MOSAT: finding safety violations of autonomous driving systems using multi-objective genetic algorithm

Tian, Haoxiang; Jiang, Yan; Wu, Guoquan; Yan, Jiren; Wei, Jun; Chen, Wei; Li, Shuo; Ye, Dan

doi:10.1145/3540250.3549100

Cited by 26 publications

(9 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The number of bugs revealed by a test suite is a common measure of assessing the quality of testing for an AI-based system [3,6,46,80]. However, for any AI-based system in general, and for AVs in particular, solely focusing on finding more bugs may overlook critical safety issues if the test suite does not cover a diverse range of scenarios, environmental conditions, and edge cases.…”

Section: Area Of Buggy Region (𝑎𝑟𝑒𝑎 𝑏𝑢𝑔𝑠mentioning

confidence: 99%

Towards Reliable AI: Adequacy Metrics for Ensuring the Quality of System-level Testing of Autonomous Vehicles

Neelofar,

Aleti

2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

AI-powered systems have gained widespread popularity in various domains, including Autonomous Vehicles (AVs). However, ensuring their reliability and safety is challenging due to their complex nature. Conventional test adequacy metrics, designed to evaluate the effectiveness of traditional software testing, are often insufficient or impractical for these systems. White-box metrics, which are specifically designed for these systems, leverage neuron coverage information. These coverage metrics necessitate access to the underlying AI model and training data, which may not always be available. Furthermore, the existing adequacy metrics exhibit weak correlations with the ability to detect faults in the generated test suite, creating a gap that we aim to bridge in this study.In this paper, we introduce a set of black-box test adequacy metrics called "Test suite Instance Space Adequacy" (TISA) metrics, which can be used to gauge the effectiveness of a test suite. The TISA metrics offer a way to assess both the diversity and coverage of the test suite and the range of bugs detected during testing. Additionally, we introduce a framework that permits testers to visualise the diversity and coverage of the test suite in a two-dimensional space, facilitating the identification of areas that require improvement.We evaluate the efficacy of the TISA metrics by examining their correlation with the number of bugs detected in system-level simulation testing of AVs. A strong correlation, coupled with the short computation time, indicates their effectiveness and efficiency in estimating the adequacy of testing AVs.

show abstract

Section: Area Of Buggy Region (𝑎𝑟𝑒𝑎 𝑏𝑢𝑔𝑠mentioning

confidence: 99%

Towards Reliable AI: Adequacy Metrics for Ensuring the Quality of System-level Testing of Autonomous Vehicles

Neelofar,

Aleti

2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

show abstract

“…To optimize these objectives, we introduce four functions: Minimum Distance Between Vehicles (MDBV) (Ben Abdessalem et al, 2016) for collision risk, Time Exposed Time-to-Collision (TET) (Mahmud et al, 2017) for interaction frequency, the Variation Rate of ego vehicle's Acceleration (VOA) (Tian et al, 2022) for driving instability, and Similarity in Trajectories and Safety Violations (STSV) for scenario diversity.…”

Section: Multi-objective Evaluationmentioning

confidence: 99%

TM-fuzzer: fuzzing autonomous driving systems through traffic management

Lin,

Chen,

et al. 2024

Preprint

View full text Add to dashboard Cite

Simulation testing of Autonomous Driving Systems (ADS) is crucial for ensuring the safety of autonomous vehicles. Currently, scenarios searched by ADS simulation testing tools are less likely to expose ADS issues and highly similar. In this paper, we propose TM-fuzzer, a novel approach for searching ADS test scenarios, which utilizes real-time traffic management and diversity analysis to search security-critical and unique scenarios within the infinite scenario space. TM-fuzzer dynamically manages traffic flow by manipulating non-player characters near autonomous vehicle throughout the simulation process to enhance the efficiency of test scenarios. Additionally, the TM-fuzzer utilizes clustering analysis on vehicle trajectory graphs within scenarios to increase the diversity of test scenarios. Compared to the baseline, the TM-fuzzer identified 29 unique violated scenarios more than four times faster and enhanced the incidence of ADS-caused violations by 26.26%. Experiments suggest that the TM-fuzzer demonstrates improved efficiency and accuracy.

show abstract

“…Furthermore as discussed below, it is a source of inconsistencies as the Scheduler's perception of the junction differs from that of a vehicle based on its own sensor equipment. # class of distance (0.01,0.01,0.01,0.01) distance (0.3,0.3,0.3,0.3) distance (20,20,20,20) distance (0.3,0.3,20,20) abstract A: speed (0,0,0,0) B: speed (0,0,0,0) C: speed (0,0,0,0) D: speed (10,10,10,10) E: speed (0,0,0,0) scenarios (20,20,20,20) distance (20,0.3,20,0.3) abstract F: speed (0,0,0,0) G: speed (0,0,0,0) H: speed (0,0,0,0) I: speed (10,10,10,10) J: speed (0,0,0,0) scenarios Lack of controllability for short distances. During initialization, the controller always randomly assigns a rate for vehicle acceleration without taking into account the safe braking distance ahead.…”

Section: 31mentioning

confidence: 99%

“…In the literature of simulation-based validation for autonomous driving systems [12,15], there is a large body of work on the generation of safety-critical scenarios, either by using scenario modeling languages [9,16], or from available databases [20].…”

Section: Related Workmentioning

confidence: 99%

“…The basic idea of the search is to perturb the driving maneuvers of traffic participants and optimize the safety constraints based on vehicle dynamics. MOSAT [20] uses genes to encode basic driving maneuvers and applies a multi-objective genetic algorithm to search for adversarial and diverse test scenarios. None of them considers violation of properties or traffic rules as the criteria in identifying safety-critical scenarios.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Simulation-based Validation for Autonomous Driving Systems

Li¹,

Sifakis²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Simulation is essential to validate autonomous driving systems. However, a simple simulation, even for an extremely high number of simulated miles or hours, is not sufficient. We need well-founded criteria showing that simulation does indeed cover a large fraction of the relevant real-world situations. In addition, the validation must concern not only incidents, but also the detection of any type of potentially dangerous situation, such as traffic violations.We investigate a rigorous simulation and testing-based validation method for autonomous driving systems that integrates an existing industrial simulator and a formally defined testing environment. The environment includes a scenario generator that drives the simulation process and a monitor that checks at runtime the observed behavior of the system against a set of system properties to be validated. The validation method consists in extracting from the simulator a semantic model of the simulated system including a metric graph, which is a mathematical model of the environment in which the vehicles of the system evolve. The monitor can verify properties formalized in a first-order linear temporal logic and provide diagnostics explaining their non satisfaction. Instead of exploring the system behavior randomly as many simulators do, we propose a method to systematically generate sets of scenarios that cover potentially risky situations, especially for different types of junctions where specific traffic rules must be respected. We show that the systematic exploration of risky situations has uncovered many flaws in the real simulator that would have been very difficult to discover by a random exploration process.

show abstract

MOSAT: finding safety violations of autonomous driving systems using multi-objective genetic algorithm

Cited by 26 publications

References 44 publications

Towards Reliable AI: Adequacy Metrics for Ensuring the Quality of System-level Testing of Autonomous Vehicles

Towards Reliable AI: Adequacy Metrics for Ensuring the Quality of System-level Testing of Autonomous Vehicles

TM-fuzzer: fuzzing autonomous driving systems through traffic management

Simulation-based Validation for Autonomous Driving Systems

Contact Info

Product

Resources

About