Comprehensive Safety Evaluation of Highly Automated Vehicles at the Roundabout Scenario

Abstract

A highly automated vehicle (HAV) is a safety-critical system. Therefore, a verification and validation (V&V) process that rigorously evaluates the safety of HAVs is necessary before their release to the market. In this paper, we propose an interaction-aware safety evaluation framework for the HAV and apply it to the roundabout entering, a highly interactive driving scenario with various traffic situations. Instead of assuming that the primary other vehicles (POVs) take predetermined maneuvers, we model the POVs as game-theoretic agents. To capture a wide variety of interactions between the POVs and the vehicle under test (VUT), we use level-k game theory and social value orientation to characterize the interactive behaviors and train a diverse library of POVs using reinforcement learning. The game-theoretic library, together with initial conditions, form a rich testing space for the two-POV roundabout scenario. On the other hand, we propose an adaptive test case generation scheme based on adaptive sampling and stochastic optimization to efficiently generate customized challenging cases for the VUT from the testing space. In simulations, the proposed testing space design captured a wide range of interactive situations at the roundabout scenario. The proposed test case generation scheme was found to cover the failure modes of the VUT more effectively compared to other test case generation approaches.

Challenge with ambiguity: Interaction-aware Corner Case Generation for the Evaluation of Highly Automated Vehicles

“…To evaluate the performance of the VUT in each test case, we define a performance score similar to our previous work [3]. The score is determined by the minimal separation between the PORU and the VUT, and the task accomplishment.…”

Challenge with ambiguity: Interaction-aware Corner Case Generation for the Evaluation of Highly Automated Vehicles

“…It states that the likelihood of observing an action is exponentially proportional to the estimated utility of that action, as expressed in Eq. (3).…”

“…The different interacting behaviors of the PORU create diverse challenges to the VUT, thus they should be considered in the test design. Evaluation for interactive scenarios is a less-studied problem, and has only been addressed by several recent studies [3], [4].…”

“…Among them, utility-based methods model human agents as rational players that behave (near) optimally according to a defined utility function. Various game-theoretic models were applied to model the interaction between human and automated vehicles in [3], [4], [27]- [29]. In [3], [29], social value orientation was applied to model the diverse value preference of the human drivers.…”

“…Various game-theoretic models were applied to model the interaction between human and automated vehicles in [3], [4], [27]- [29]. In [3], [29], social value orientation was applied to model the diverse value preference of the human drivers. In [30], cumulative prospect theory was applied to model the irrationality of human driving behaviors.…”