Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validatio

Corso, Anthony; Du, Peter; Driggs-Campbell, Katherine; Kochenderfer, Mykel J.

doi:10.1109/itsc.2019.8917242

Cited by 73 publications

(64 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The advantage of Reinforcement Learning is that it can even change the time signals during run-time based on the assessment results in the current time step within the scenario execution. Corso et al [128] develop the approach further with a reward-augmentation technique. Both papers build on a predecessor paper [129] from the avionic domain.…”

Section: Simulation-based Falsificationmentioning

confidence: 99%

Survey on Scenario-Based Safety Assessment of Automated Vehicles

et al. 2020

View full text Add to dashboard Cite

When will automated vehicles come onto the market? This question has puzzled the automotive industry and society for years. The technology and its implementation have made rapid progress over the last decade, but the challenge of how to prove the safety of these systems has not yet been solved. Since a market launch without proof of safety would neither be accepted by society nor by legislators, much time and many resources have been invested into safety assessment in recent years in order to develop new approaches for an efficient assessment. This paper therefore provides an overview of various approaches, and gives a comprehensive survey of the so-called scenario-based approach. The scenario-based approach is a promising method, in which individual traffic situations are typically tested by means of virtual simulation. Since an infinite number of different scenarios can theoretically occur in real-world traffic, even the scenario-based approach leaves the question unanswered as to how to break these down into a finite set of scenarios, and find those which are representative in order to render testing more manageable. This paper provides a comprehensive literature review of related safety-assessment publications that deal precisely with this question. Therefore, this paper develops a novel taxonomy for the scenario-based approach, and classifies all literature sources. Based on this, the existing methods will be compared with each other and, as one conclusion, the alternative concept of formal verification will be combined with the scenario-based approach. Finally, future research priorities are derived.

show abstract

Section: Simulation-based Falsificationmentioning

confidence: 99%

Survey on Scenario-Based Safety Assessment of Automated Vehicles

et al. 2020

View full text Add to dashboard Cite

show abstract

“…DRL has shown state-of-the-art results in playing Atari games (Mnih et al, 2015), playing chess (Silver et al, 2017), and robot manipulation from camera input (Gu et al, 2017). In recent years, different DRL techniques have been applied to falsification and most-likely failure analysis (Akazaki et al, 2018;Behzadan & Munir, 2019;Corso et al, 2019;Koren et al, 2018;Kuutti et al, 2020;Qin et al, 2019).…”

Section: Deep Reinforcement Learningmentioning

confidence: 99%

A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems

CorsoAnthony¹,

Moss

KorenMark

et al. 2021

jair

Self Cite

View full text Add to dashboard Cite

Autonomous cyber-physical systems (CPS) can improve safety and efficiency for safety-critical applications, but require rigorous testing before deployment. The complexity of these systems often precludes the use of formal verification and real-world testing can be too dangerous during development. Therefore, simulation-based techniques have been developed that treat the system under test as a black box operating in a simulated environment. Safety validation tasks include finding disturbances in the environment that cause the system to fail (falsification), finding the most-likely failure, and estimating the probability that the system fails. Motivated by the prevalence of safety-critical artificial intelligence, this work provides a survey of state-of-the-art safety validation techniques for CPS with a focus on applied algorithms and their modifications for the safety validation problem. We present and discuss algorithms in the domains of optimization, path planning, reinforcement learning, and importance sampling. Problem decomposition techniques are presented to help scale algorithms to large state spaces, which are common for CPS. A brief overview of safety-critical applications is given, including autonomous vehicles and aircraft collision avoidance systems. Finally, we present a survey of existing academic and commercially available safety validation tools.

show abstract

“…Meanwhile, falsification-based evaluation methods attempt to generate initial conditions or POV behaviors that force the VUT to violate the safety requirements with limited test runs. Corner cases have been generated using simulated annealing [23], rapid-exploring random tree (RRT) [24], evolutionary algorithms [25], Bayesian optimization [26], adaptive sampling [27], reinforcement learning [28], [29], etc. In [23], [24], the falsifying POVs might behave adversarially, which is not a reasonable representation of real-driving situations.…”

Section: A Scenario-based Safety Evaluation For Havsmentioning

confidence: 99%

Comprehensive Safety Evaluation of Highly Automated Vehicles at the Roundabout Scenario

Wang¹,

Zhang²,

Peng³

2021

Preprint

View full text Add to dashboard Cite

A highly automated vehicle (HAV) is a safety-critical system. Therefore, a verification and validation (V&V) process that rigorously evaluates the safety of HAVs is necessary before their release to the market. In this paper, we propose an interaction-aware safety evaluation framework for the HAV and apply it to the roundabout entering, a highly interactive driving scenario with various traffic situations. Instead of assuming that the primary other vehicles (POVs) take predetermined maneuvers, we model the POVs as game-theoretic agents. To capture a wide variety of interactions between the POVs and the vehicle under test (VUT), we use level-k game theory and social value orientation to characterize the interactive behaviors and train a diverse library of POVs using reinforcement learning. The game-theoretic library, together with initial conditions, form a rich testing space for the two-POV roundabout scenario. On the other hand, we propose an adaptive test case generation scheme based on adaptive sampling and stochastic optimization to efficiently generate customized challenging cases for the VUT from the testing space. In simulations, the proposed testing space design captured a wide range of interactive situations at the roundabout scenario. The proposed test case generation scheme was found to cover the failure modes of the VUT more effectively compared to other test case generation approaches.

show abstract

Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validatio

Cited by 73 publications

References 12 publications

Survey on Scenario-Based Safety Assessment of Automated Vehicles

Survey on Scenario-Based Safety Assessment of Automated Vehicles

A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems

Comprehensive Safety Evaluation of Highly Automated Vehicles at the Roundabout Scenario

Contact Info

Product

Resources

About