As AI and IoT technologies continue to evolve, edge devices are becoming increasingly sophisticated, transforming from basic data collectors into real-time, decision-making systems. One significant driver of this shift is the adoption of large-scale language models (LLMs) which enable devices to process and analyze data locally. According to IMARC Group, the global edge AI market reached $15.4 billion in 2023 and is projected to surge to $70.9 billion by 2032, at a CAGR of 18.5%.
This booming demand introduces challenges, including the complexity of integrating diverse systems, latency from cloud-based LLMs, and rising costs in hardware and maintenance. Cloud-based LLMs fall short in edge environments due to latency and security concerns. To address these challenges, DFI, in collaboration with Intel, Canonical and other key partners, has developed the DFI Workload Platform—an innovative solution that combines multi-OS architecture with multi-functions tailored for x86-based edge AI computing applications for diverse industries.
Industry: Smart Retail, Smart Transportation, EV Charging, Smart Factory and more
Application:Smart Retail Hospitality, Airport Self-Service Kiosk, Smart EV Charging Station, Smart Factory Assembly line, Industrial AMR and more
Solution: DFI Workload Platform
Key Features of DFI Workload Platform, including:
In 2023, Bloomberg reported a security breach in which a leading Korean semiconductor company discovered an accidental leak of sensitive internal source code by an employee who uploaded it to ChatGPT. Incidents like these highlight the privacy risks of cloud-based generative AI, where companies face limited control over uploaded data. Additionally, AI models also often produce unreliable or fabricated information due to "hallucinations," further complicating efforts to ensure both data security and accuracy..
Cloud-based LLMs require constant data transfer between devices and cloud servers, leading to latency, especially under network constraints. For applications like factory automation or self-service kiosks, even slight delays in real-time responses can disrupt operations.
Implementing LLMs involves extensive costs, including model development, token usage, and operational expenses for hardware, cabling, and maintenance. High-performance computing (HPC) demands, data acquisition, storage, and pre-processing further increase the technical and financial burden. While cloud solutions offer scalability and flexibility, long-term expenses tied to infrastructure expansion, maintenance, and cumulative token consumption can add up significantly over time.
The energy consumption of cloud-based AI infrastructure is staggering, with IDC reporting that AI data centers consumed approximately 400 terawatt hours (TWh) in 2023. Gartner predicts that by 2025, AI could consume more energy than the entire human workforce, highlighting sustainability challenges for cloud-based AI systems.
The DFI Workload Platform, built on Intel’s x86 architecture, effectively addresses several challenges in edge AI deployment. It enables direct deployment of trained AI models and software onto the platform, even across multiple OS. This edge-based solution delivers enhanced energy efficiency, cost-effectiveness, and security while ensuring real-time performance, making it ideal for edge applications that demand rapid response times and high performance.
By leveraging the latest Intel Virtualization Technology, the DFI Workload Platform overcomes the traditional latency issues of hypervisors and delivers a real-time, multi-OS platform. It integrates computing, storage, and networking resources into a single system, utilizing Intel iGPU SR-IOV to efficiently allocate GPU resources across multiple virtual machines (VMs). This optimization enables various applications, even AI-driven applications, to run stably and in real-time on a hypervisor.
Notably, DFI is the first IPC manufacturer to successfully implement SR-IOV in real-world applications.
In partnership with Intel, Canonical and other key partners, the DFI Workload Platform ensures extensive OS compatibility, supporting Windows, Ubuntu, Android, and Linux. This flexibility not only enhances usability but also strengthens robust system security, making it suitable for diverse edge computing scenarios.
DFI Workload Platform integrates Large Language Models (LLMs) as On-Device LLMs, enabling AI inference and computing directly on local edge devices for high-performance and low-latency operations. Leveraging the powerful combination of the Intel® Arc™ GPU, Intel® Core™ processor, and OpenVINO™ toolkit, this platform maximizes hardware efficiency. With the OpenVINO™ toolkit for Intel® Arc™ GPU optimizations, complex LLMs can run under 6 GB of memory, while offloading intensive AI tasks to the GPU reduces response time by up to 66% and supports multiple concurrent workloads. Along with virtualization technology and interactive digital signages, multiple LLM-driven self-service applications can run seamlessly in a single device across multiple OS to provide interactive user experience. This innovative approach is reshaping the edge AI computing industry, delivering highly personalized, engaging, and revenue-generating experiences.
Discover more about the Intel® Arc™ GPU for the Edge through DFI’s EV charging station success case.
Since all AI processing occurs on one edge device, On-Device LLMs are ideal for a variety of application scenarios, like retail stores, kiosks, and EV chargers, where data scale and security are paramount. On-Device LLMs leverage application-specific data and training models to optimize performance for each scenario while minimizing the need for extensive databases, which in turn reduces latency and enhances data privacy—crucial for handling sensitive customer information. This adaptable design facilitates real-time AI processing tailored to various applications.
The Intel® OpenVINO™ toolkit is a top-tier solution for optimizing and deploying LLMs on end-user systems and devices. It enables developers to compress large models, seamlessly integrate them into AI-powered applications, and deploy them on edge devices for maximum performance. This ensures efficient, high-speed inference, ideal for applications requiring real-time AI capabilities.
So far, DFI has successfully implemented the DFI Workload Platform in various applications across the Asia-Pacific region, including:
Integrating multiple functions into a single DFI’s systems, including facial recognition via computer vision, digital signage, smart ads, On-Device LLMs serving as intelligent assistants (IA) for in-store shopping, and self-checkout POS systems to streamline store operations and enhance smart retail efficiency.
Serving as a tool for passenger information inquiries by quickly providing guidance on terminal facilities and real-time boarding information through On-Device LLMs, along with digital signage and computer vision like crowd monitoring, all powered by a single embedded system from DFI
Integrating digital payment systems, digital signage, and LLM technology, enabling users to discover surrounding services and information seamlessly while utilizing a DFI’s Mini-ITX industrial motherboard.
Using multi-OS architecture to seamlessly integrate various application of machine vision, motions control, robotuic controller, slam board, Time of Flight(ToF), on a x86 device, enhancing Industrial AMR operational efficiency by DFI’s Single Board Computer (SBC). Customers choose our solution because it significantly reduces Total Cost of Ownership (TCO).
Each of the aforementioned scenarios utilizes the DFI Workload Platform to meet the diverse application needs of customers.
The multi-OS architecture simplifies system integration by supporting a wide range of operating systems. This reduces the barriers associated with integrating multiple application systems, saving significant time and effort in building a unified platform that can accommodate all connected applications to operate seamlessly.
Integrating multiple applications on a single x86 device greatly reduces the number of required hardware components, resulting in a significant decrease in TCO. By consolidating to a single hardware unit, many hidden costs associated with multiple devices—such as wiring, switch expenses and operational as well as maintenance costs—are greatly minimized. Applications also benefit from enhanced computing efficiency through faster Virtual LAN performance and shared memory access.Additionally, using On-Device LLMs helps avoid the high costs associated with cloud-based LLMs, eliminating the need for costly high-bandwidth data transfers and ongoing cloud service subscriptions.
Additionally, DFI OOB solution can work with the DFI Workload Platform to reduce device management costs by facilitating remote management of devices. It allows remote power control when edge devices are unresponsive or shutdown, supports BIOS update, and allocates the boot partition to switch OS before system restart, in order to minimize downtime.
The DFI Workload Platform maximizes device utility while improving PPW. In a PPW test, the x86-based EC70A-TGU-i7-1185GRE system—running four virtual machines—demonstrated lower power consumption compared to four ARM-based NXP i.MX8 systems, despite their known energy efficiency. Moreover, according to GeekBench 5 benchmark results, the x86 platform achieved impressive single-core and multi-core scores of 108 and 125, respectively, in contrast to only 20 and 68 for the ARM platform. This underscores the x86 architecture's potential for superior energy efficiency when combined with the DFI Workload Platform.
The powerful multi-OS architecture and On-Device LLMs bring values and business opportunities to a wide range of industries. With On-Device LLMs, devices can deliver personalization and customization information to customers or workers based on specific scenarios. Taking Smart Retail and Smart Factory for example:
AI-powered computer vision enables real-time customer identification, such as facial, age or gender recognitions, allowing Content Management Systems(CMS) & On-Device LLMs to deliver personalized advertisements and exclusive offers to VIPs, thereby enhancing the shopping experience.
Smart edge-based systems offer AI-driven voice interactions to assist customers with product inquiries, in-store navigation, and self-checkout. This reduces reliance on staff and boosts service efficiency.
Retailers can leverage computer vision with the DFI Workload Platform for swift inventory and sales trend analysis, enabling quick adjustments to product displays and promotional strategies to improve sales effectiveness.
Local processing of computer vision n and customer behavior data enhances privacy and mitigates risks associated with cloud data transfers, fostering greater customer trust.
Integrating multiple applications on a single device, rather than relying on multiple devices, reduces labor and operational costs while enhancing efficiency, especially in AMR application for heavy industry assembly environments.
On-Device LLMs serve as intelligent assistants (IA), allowing workers to search production-related and internal information like SOPs, announcement, yield...etc. Workers can also manage production schedules via voice commands, with data security ensured as everything runs locally on the device.
Well-trained On-Device LLMs in manufacturing settings can identify potential risks and issues in real-time to optimize supply chain operations. Since all information is processed locally, sensitive data remains confidential and secure.
Edge AI data analysis systems monitor equipment performance in real-time and use historical data and machine learning models to predict potential malfunctions, allowing proactive maintenance and minimizing downtime.
This article aims to highlight two key concepts: the innovative Edge AI computing approaches embodied by the DFI Workload Platform, and the critical role of On-Device LLMs.
In an era where computing power is the new oil, edge AI computing has become essential. Nvidia’s recent rise in market value exemplifies the surging demand for computing power. However, as mentioned earlier, increased computational needs also mean rising costs, as more users drive higher expenses. In the x86 ecosystem, DFI has developed a long-standing partnership with Intel and other key partners. Through this collaboration, DFI has enhanced its Workload Platform through virtualization technology and is actively engaging with customers on a variety of innovative projects.
The DFI Workload Platform revolutionizes edge AI computing by leveraging On-Device LLMs and a multi-OS architecture to meet the increasing demands of AI and IoT. By allowing multiple applications or software to run on a single x86 device, it empowers industries like retail, stores and smart manufacturing to unlock new business opportunities and improve operational efficiency. As the edge AI market is projected to grow significantly, solutions like the DFI Workload Platform will be crucial for organizations aiming to innovate and maintain competitiveness.
DFI is the world’s leading brand in embedded motherboards and industrial computers. With over 40 years of expertise in embedded systems, we specialize in Edge AI computing, leveraging our extensive experience to design and develop high-performance hardware solutions tailored to meet the evolving demands of modern industries. Our solutions ensure seamless integration within edge environments, delivering outstanding efficiency and reliability. Through the DFI Workload Platform and DFI OOB solution, we facilitate smooth collaboration between hardware and AI applications, providing peak performance and operational excellence to our customers.