Ahead of next month’s ADAS & Autonomous Vehicle Technology Expo California, Waymo’s senior research scientist at the Safety Research and Best Practices team, Azadeh Dinparastdjadid, delves into how behavior evaluation is an important component of ADS safety.
Why is behavioral capability in regulatory compliance, conflict avoidance and collision avoidance important for ADS safety?
On the path to ensuring ADS safety we need to demonstrate the absences of unreasonable risk. The term ‘absence of unreasonable risk’ can be further decomposed into risk subcomponents, including ‘behavioral hazards’, which can be defined as those hazards associated with potential sources of harm resulting from ADS driving behavior, whether intended or unintended/unforeseen. An example of this is the ADS having an undesired degree of proximity to surrounding road users. Along these lines, ensuring sufficient behavioral capability in all three areas (regulatory compliance, conflict avoidance and collision avoidance) is key to eliminating behavioral hazards.
Why do the annexes in ISO, 2022 highlight the need for more detailed scenario-dependent behavioral evaluation around conflict avoidance specifically?
The annexes in ISO, 2022 (Road Vehicles – safety of the intended functionality ISO 21448:2022), in particular the SOTIF standard’s Annex D.1 on driving policies, discuss the set of rules required to navigate safely. The general aim of these guidelines is to ensure that the ADS exhibits sufficiently defensive driving to avoid getting involved in conflicts in the first place, which is what we call conflict avoidance.
What computational metrics can be used as behavioral benchmarks for flagging when the ADS’s behavior does not meet expectations?
To answer this question, it is worth considering what is meant by expectations. Alignment on what these societal ‘expectations’ are and what principles these metrics should address is crucial. At Waymo, we have introduced the concept of drivership – the realization of good driving behavior, predicated on the alignment of behavior and societal norms and expectations, that positions the ADS as a good citizen of the road. One approach to operationalizing drivership is through behavioral benchmarks or reference behavior models.
In my presentation, ‘Automated driving system behavior evaluation, I will talk about some of the behavior reference models we have developed at Waymo, including The Non-Impaired Eyes ON Model (NIEON) Conflict Driver, Surprise and the Field of Safe Motion (FSM).
How can they also be applied to discovering novel scenarios and events with behavioral issues from existing datasets?
If we define a behavioral reference model as any metric or criterion that can be used as a benchmark to operationalize proper/improper drivership, such models can then also be used as a tool to discover and flag instances of improper drivership in existing datasets.
Why did you decide to choose to speak on this subject at ADAS & Autonomous Vehicle Technology Expo California?
When it comes to the acceptance and adoption of AV technology, the research concept of drivership can build trust by meeting and exceeding societal expectations on how autonomous driving systems behave on the road. Our team’s work on models such as NIEON, Surprise and FSM will help operationalize drivership, which encapsulates what it means to be a good citizen on the road. We are also leading a new SAE-recommended practice on behavioral reference models for automated driving systems which aligns with the topic of this presentation.