FEV is collaborating with Microsoft to integrate in-vehicle generative AI capabilities built on Nvidia GPU-accelerated compute and AI model microservices. The cooperation aims to enable multimodal voice, text and gesture interactions directly in the vehicle – independent of a permanent internet connection.
The focus is on the use of small language models (SLM) such as Microsoft’s Phi-4-mini-instruct in Microsoft Foundry, which is powered by Nvidia Drive AGX accelerated compute. The solution allows vehicle functions such as the dashboard or individual vehicle profiles to be configured by voice command. At the same time, the system acts as robust local backup intelligence for cloud-based large language models (LLMs).
More intelligence, robustness and efficiency
Auto makers are increasingly deploying small language models (SLMs) directly on vehicle hardware rather than relying solely on cloud-based processing. Because inference happens locally, core functions can remain available with limited or no internet connectivity. This approach can also reduce reliance on cloud infrastructure and backend costs, since embedded SLMs can supplement or, in some cases, replace cloud-based large language models depending on the specific use case, potentially making it more cost-effective for auto makers to scale software-defined vehicle features.
“Our collaboration with Microsoft and Nvidia showcases how small, efficient language models can transform in-vehicle experiences, delivering powerful functionality without the overhead of larger systems,” said Thomas Hülshorst, group vice president of intelligent mobility and software at FEV.
“By combining advanced AI frameworks with domain- and task-specific optimizations, FEV and Microsoft are shaping the future of intelligent, voice-driven interfaces that meet the high standards of automotive deployment,” added Boris Scholl, vice president of engineering at Microsoft.
Central fields of application for embedded Gen AI
As part of its collaboration with Microsoft and Nvidia, FEV is exploring several application areas it sees as having strong commercial potential. These include automated and autonomous driving (SAE Levels 3-5), where multimodal generative AI models are intended to improve recognition of objects, traffic situations and driving paths, particularly in complex urban environments and edge cases. Another focus is driver and passenger monitoring, where embedded generative AI aims to improve detection of fatigue, distraction or unusual behavior, with local processing intended to serve as a backup to cloud-based systems for safety-relevant functions. A third area is personalized vehicle and HMI configuration, where voice commands would allow vehicle functions and interfaces to be adapted for different driver profiles or use cases without relying on external cloud infrastructure.
Multimodal system architecture on Nvidia platforms
The underlying architecture is designed as a multimodal system and processes speech, text and visual information. To achieve high performance in the constrained environment of an embedded in-vehicle environment, FEV has used synthetically generated data curated with Nvidia NeMo in the fine-tuning process to optimize the Phi-4-mini-instruct model. This is followed by integration and deployment of the resulting model running on Nvidia Drive AGX, enabling further improvement in model performance. The AI functions are operated as modular software services inside the vehicle.
Personalized vehicle and HMI configuration
Building on these technologies, FEV has developed a dashboard configurator showcase that highlights the potential of intelligent in-vehicle interfaces. Using natural voice commands, a locally deployed small language model dynamically updates the dashboard, reducing reliance on continuous cloud connectivity while enabling fast, responsive performance.
The showcase demonstrates strong real-time performance. In the future, FEV plans to gradually supplement or replace cloud-based AI functions via models that run locally in the vehicle.
In related news, hands-free driving and smarter navigation added to Lucid Gravity