Rapid AI advances now enable engineers to develop autonomous driving technology faster than ever, but the true frontier of autonomous driving is the ability to couple those advances with demonstrable and rigorous safety.
Progress in autonomous vehicle development no longer depends on larger budgets or fleets, but on extracting meaningful insights from real-world testing and simulation. The focus is on validating how autonomous systems respond to rare and complex scenarios that may occur only infrequently in real-world driving.
To meet this challenge, Kodiak says it has adopted two tools that accelerate the pace, depth and precision of safety engineering to deliver clear, compelling evidence of the safety of Kodiak Driver.
The first tool is Kodiak’s Probabilistic Risk Assessment (PRA), a methodology that the company uses to estimate the expected rate of collisions of varying severities for the Kodiak Driver and to identify the key scenarios, risk factors and autonomy failure modes that dominate the risk profile. The company then compares this output against human performance baselines, which are established in partnership with leading centers of transportation research.
The second tool is BreakPoint, an artificial intelligence (AI) validation tool that the company says hunts with intelligence and efficiency for edge cases that could result in collisions or other undesirable behavior.
The analysis provided by BreakPoint helps inform the company’s PRA models, enabling Kodiak to identify key risk areas for the Kodiak Driver and prioritize development efforts accordingly. Together, these tools form part of the company’s safety case and support the development and deployment of its AI-powered autonomous driving system.
Quantifying safety
The Kodiak PRA combines Bayesian probability theory, systems engineering, reliability analysis and statistical modeling to quantify safety performance. It is designed to estimate collision rates for rare scenarios that are difficult to capture through real-world testing alone, while also quantifying the uncertainty associated with those estimates. This enables the company to identify where evidence is strong and where additional validation is needed.
The Kodiak PRA breaks down operating scenarios into three key factors: scenario exposure, which measures how often the vehicle encounters a particular situation; collision likelihood, which estimates the probability of a collision occurring when that scenario is encountered; and collision severity, which assesses the potential consequences of a collision in that scenario.
The PRA provides a quantitative framework for assessing known hazardous scenarios and evaluating their associated risks. BreakPoint complements this by identifying previously unknown or untested hazardous scenarios, bringing them into the scope of analysis. Together, the tools support a systematic approach to identifying, quantifying and mitigating safety risks, helping to address the challenges outlined in the Safety of the Intended Functionality (SOTIF) framework.
Risk-informed prioritization
With the PRA results, Kodiak directly compares the Kodiak Driver against the rate of collisions from comparable human drivers in analogous operating environments. By breaking down driving into structured events ranging from the mundane to rare, the company says it can estimate risk with bounded uncertainty and ensure the Kodiak Driver achieves superior performance.
Beyond deployment decisions, the PRA helps identify and prioritize key risk factors throughout development. Its Bayesian framework highlights where additional evidence is needed, helping to focus testing and validation on less-explored scenarios.
BreakPoint
A common engineering testing technique is to deliberately inject faults into a system and observe the system’s response, to ensure it can handle the fault as expected.
BreakPoint introduces realistic, time-varying errors into the autonomy system to evaluate performance under normal and challenging conditions. The tool uses an adversarial approach to identify potential failure modes by searching for scenarios that could lead to collisions. These findings are then assessed for risk and incorporated into the PRA framework to support ongoing safety analysis.
According to Kodiak, BreakPoint has identified previously unknown failure modes that had not been observed during real-world testing. One example involved a low-probability scenario in which the perception system could incorrectly estimate the speed of stalled vehicles within its industrial operating domain. The company said identifying the issue in simulation enabled it to address the scenario more quickly, noting that it may have required many thousands of miles of real-world driving to encounter the same event.
In related news, Volvo and Boliden move 700,000 metric tons of rock material with autonomous trucks in dam construction project