SceGene: Bio-Inspired Traffic Scenario Generation for Autonomous Driving Testing

Li, Ao; Chen, Shitao; Sun, Liting; Zheng, Nanning; Tomizuka, Masayoshi; Zhan, Wei

doi:10.1109/tits.2021.3134661

Cited by 15 publications

(4 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to [256], our main aim is to show the ecosystems and the landscape of the use of these languages in ADS testing, as a reference for the readers to better understand the testing techniques in §6 and §7. Also, our study includes some latest achievements, e.g., paracosm [255] and SceGene [254], in this direction.…”

Section: Programming Languages For Scenario Generationmentioning

confidence: 99%

“…As shown by Table 12, we collect 8 representative programming languages, including the classic ones, such as OpenScenario, that have been widely used in different stages of the development of ADS, and emerging ones, such as paracosm [255]. As our findings, first, different languages are designed for different purposes and attached with different features, e.g., Scenic [250] allows probabilistic sampling for testing driving systems with machine learning components; SceGene [254] designs bio-inspired operations, such as crossover, mutation, for scenario generation. Second, some Including a perception dataset with high-resolution sensor data and labels, and a motion dataset with object trajectory and corresponding 3D map Huge videos [225] 2020…”

Section: Programming Languages For Scenario Generationmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Automated Driving System Testing: Landscapes and Trends

Tang¹,

Zhang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Automated Driving Systems (ADS) have made great achievements in recent years thanks to the efforts from both academia and industry. A typical ADS is composed of multiple modules, including sensing, perception, planning and control, which brings together the latest advances in multiple domains. Despite these achievements, safety assurance of the systems is still of great significance, since the unsafe behavior of ADS can bring catastrophic consequences and unacceptable economic and social losses. Testing is an important approach to system validation for the deployment in practice; in the context of ADS, it is extremely challenging, due to the system complexity and multidisciplinarity. There has been a great deal of literature that focuses on the testing of ADS, and a number of surveys have also emerged to summarize the technical advances. However, most of these surveys focus on the system-level testing that is performed within software simulators, and thereby ignore the distinct features of individual modules. In this paper, we provide a comprehensive survey on the existing ADS testing literature, which takes into account both module-level and system-level testing. Specifically, we make the following contributions: (1) we build a threat model that reveals the potential safety threats for each module of an ADS; (2) we survey the module-level testing techniques for ADS and highlight the technical differences affected by the properties of the modules; (3) we also survey the system-level testing techniques, but we focus on empirical studies that take a bird's-eye view on the system, the problems due to the collaborations between modules, and the gaps between ADS testing in simulators and real world; (4) we identify the challenges and opportunities in ADS testing, which facilitates the future research in this field. CCS Concepts: • Computer systems organization → Embedded systems; • Software and its engineering → Software verification and validation; • Security and privacy → Systems security; • Computing methodologies → Artificial intelligence.

show abstract

Section: Programming Languages For Scenario Generationmentioning

confidence: 99%

Section: Programming Languages For Scenario Generationmentioning

confidence: 99%

A Survey on Automated Driving System Testing: Landscapes and Trends

Tang¹,

Zhang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this context, researchers have proposed many violated scenario search methods (Han and Zhou, 2020;Haq et al, 2022;Zhou et al, 2023a,b), such as guided searching methods (Li et al, 2020;Cheng et al, 2023;Tang et al, 2023), data-based methods Li et al (2022); Hou et al (2023); Priisalu et al (2022), agent-based methods (Huai et al, 2023;Kim et al, 2022;Feng et al, 2023). These methods involve identifying security-critical scenarios within an infinite scenario space that are likely to result in violations, through various risk assessment indicators (e.g., vehicle distance) (Li et al, 2020), and conducting ADS simulation tests.…”

Section: Introductionmentioning

confidence: 99%

TM-fuzzer: fuzzing autonomous driving systems through traffic management

Lin,

Chen,

et al. 2024

Preprint

View full text Add to dashboard Cite

Simulation testing of Autonomous Driving Systems (ADS) is crucial for ensuring the safety of autonomous vehicles. Currently, scenarios searched by ADS simulation testing tools are less likely to expose ADS issues and highly similar. In this paper, we propose TM-fuzzer, a novel approach for searching ADS test scenarios, which utilizes real-time traffic management and diversity analysis to search security-critical and unique scenarios within the infinite scenario space. TM-fuzzer dynamically manages traffic flow by manipulating non-player characters near autonomous vehicle throughout the simulation process to enhance the efficiency of test scenarios. Additionally, the TM-fuzzer utilizes clustering analysis on vehicle trajectory graphs within scenarios to increase the diversity of test scenarios. Compared to the baseline, the TM-fuzzer identified 29 unique violated scenarios more than four times faster and enhanced the incidence of ADS-caused violations by 26.26%. Experiments suggest that the TM-fuzzer demonstrates improved efficiency and accuracy.

show abstract

“…Testing and evaluation are major challenges for the development and deployment of connected and automated vehicles (CAVs). The past few years have witnessed increasingly rapid advances in the field of testing scenario library generation (TSLG) [1]- [12]. The goal of TSLG is usually to purposely generate safety-critical testing scenarios that can improve the evaluation efficiency of CAVs while ensuring the evaluation unbiasedness.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Testing for Connected and Automated Vehicles with Sparse Control Variates in Overtaking Scenarios

Yang¹,

He²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Testing and evaluation is a critical step in the development and deployment of connected and automated vehicles (CAVs). Due to the black-box property and various types of CAVs, how to test and evaluate CAVs adaptively remains a major challenge. Many approaches have been proposed to adaptively generate testing scenarios during the testing process. However, most existing approaches cannot be applied to complex scenarios, where the variables needed to define such scenarios are high dimensional. Towards filling this gap, the adaptive testing with sparse control variates method is proposed in this paper. Instead of adaptively generating testing scenarios, our approach evaluates CAVs' performances by adaptively utilizing the testing results. Specifically, each testing result is adjusted using multiple linear regression techniques based on control variates. As the regression coefficients can be adaptively optimized for the CAV under test, using the adjusted results can reduce the estimation variance, compared with using the testing results directly. To overcome the high dimensionality challenge, sparse control variates are utilized only for the critical variables of testing scenarios. To validate the proposed method, the high-dimensional overtaking scenarios are investigated, and the results demonstrate that our approach can further accelerate the evaluation process by about 30 times.

show abstract

SceGene: Bio-Inspired Traffic Scenario Generation for Autonomous Driving Testing

Cited by 15 publications

References 43 publications

A Survey on Automated Driving System Testing: Landscapes and Trends

A Survey on Automated Driving System Testing: Landscapes and Trends

TM-fuzzer: fuzzing autonomous driving systems through traffic management

Adaptive Testing for Connected and Automated Vehicles with Sparse Control Variates in Overtaking Scenarios

Contact Info

Product

Resources

About