WorldGen-1, a multisensor generative AI foundation model designed for simulating an entire autonomous vehicle stack, has been introduced by Helm.ai, a provider of AI software for ADAS, autonomous driving and robotics.
The model has been designed to synthesize realistic sensor and perception data across multiple modalities and perspectives. It can also extrapolate sensor data from one modality to another, and predict the behavior of the ego vehicle and other agents in the driving environment. These simulation capabilities will simplify the development and validation of autonomous driving systems, according to Helm.ai.
WorldGen-1 leverages generative deep neural network (DNN) architectures and deep teaching, an unsupervised training technology. It is trained on thousands of hours of driving data, encompassing every layer of the autonomous driving stack, including vision, perception, lidar and odometry.
The model generates realistic sensor data for surround-view cameras, semantic segmentation at the perception layer, lidar front view, lidar bird’s-eye view and the ego vehicle path in physical coordinates. By generating data across the entire AV stack, the system aims to accurately replicate potential real-world situations from the perspective of the self-driving vehicle.
“Combining innovation in generative AI architectures with our deep teaching technology yields a highly scalable and capital-efficient form of generative AI. With WorldGen-1, we aim to close the sim-to-real gap for autonomous driving, streamlining and unifying the development and validation of high-end ADAS and L4 systems. This tool is intended to accelerate development, improve safety and reduce the gap between simulation and real-world testing,” said Helm.ai’s CEO and co-founder, Vladislav Voroninski.
Additionally, the system can extrapolate from real camera data to multiple other modalities. This feature is designed to ensure the augmentation of existing camera-only datasets into synthetic multisensor datasets.
Beyond sensor simulation and extrapolation, WorldGen-1 can also predict the behaviors of pedestrians, vehicles and the ego vehicle in relation to the surrounding environment. This will enable the AI to create a wide range of potential scenarios, including rare corner cases and model multiple potential outcomes based on observed input data.
Voroninski added, “Generating data from WorldGen-1 is like creating a vast collection of diverse digital siblings of real-world driving environments, complete with smart agents that think and predict like humans, enabling us to tackle the most complex challenges in autonomous driving.”