A scenario generation pipeline for autonomous vehicle simulators

Wen, Mingyun; Park, Jisun; Cho, Kyungeun

doi:10.1186/s13673-020-00231-z

Cited by 20 publications

(11 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning (DL) models are also introduced into the generation in [28] and [29]. [28] inputs the current state of the AV and a high-definition map to a Long Short Term Memory (LSTM) [98] module to sequentially generate the trajectory of surrounding vehicles and pedestrians.…”

Section: Deep Learning Modelsmentioning

confidence: 99%

“…They train their model with normal traffic data since their target is to generate naturalistic scenarios. [29] proposes a quite complex system to generate scenarios in a simulator, which uses Convolution Neural Network (CNN) [99] as a selector to generate agents surrounding the AV. In [100], both Gated Recurrent Unit and CNN are used to learn the behaviors of multi-agents from real-world data.…”

Section: Deep Learning Modelsmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous driving systems have witnessed a significant development during the past years thanks to the advance in machine learning-enabled sensing and decision-making algorithms. One critical challenge for their massive deployment in the real world is their safety evaluation. Most existing driving systems are still trained and evaluated on naturalistic scenarios collected from daily life or heuristically-generated adversarial ones. However, the large population of cars, in general, leads to an extremely low collision rate, indicating that the safety-critical scenarios are rare in the collected real-world data. Thus, methods to artificially generate scenarios become crucial to measure the risk and reduce the cost. In this survey, we focus on the algorithms of safety-critical scenario generation in autonomous driving. We first provide a comprehensive taxonomy of existing algorithms by dividing them into three categories: data-driven generation, adversarial generation, and knowledge-based generation. Then, we discuss useful tools for scenario generation, including simulation platforms and packages. Finally, we extend our discussion to five main challenges of current works -fidelity, efficiency, diversity, transferability, controllability -and research opportunities lighted up by these challenges.

show abstract

Section: Deep Learning Modelsmentioning

confidence: 99%

Section: Deep Learning Modelsmentioning

confidence: 99%

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding¹,

Xu²,

Arief³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, such peril has been identified in different evaluation approaches [63,64,31,29,32,30], which have been already adopted by industry [1] and test agencies [43] in the U.S. to assess the safety of AVs. Second, although several learning-based scenario generation approaches are later proposed to overcome the above challenge [41,55,8,57], existing evaluation tools and platforms are usually based on their own design, such as dataset selection, safety-critical scenario definition and generation, evaluation metrics, and input types. This makes it very challenging to fairly compare different AD algorithms or interpret different evaluation results.…”

Section: Introductionmentioning

confidence: 99%

SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehicles

Xu¹,

Ding²,

Weijie³

et al. 2022

Preprint

View full text Add to dashboard Cite

As shown by recent studies, machine intelligence-enabled systems are vulnerable to test cases resulting from either adversarial manipulation or natural distribution shifts. This has raised great concerns about deploying machine learning algorithms for real-world applications, especially in the safety-critical domains such as autonomous driving (AD). On the other hand, traditional AD testing on naturalistic scenarios requires hundreds of millions of driving miles due to the high dimensionality and rareness of the safety-critical scenarios in the real world. As a result, several approaches for autonomous driving evaluation have been explored, which are usually, however, based on different simulation platforms, types of safety-critical scenarios, scenario generation algorithms, and driving route variations. Thus, despite a large amount of effort in autonomous driving testing, it is still challenging to compare and understand the effectiveness and efficiency of different testing scenario generation algorithms and testing mechanisms under similar conditions. In this paper, we aim to provide the first unified platform SafeBench to integrate different types of safety-critical testing scenarios, scenario generation algorithms, and other variations such as driving routes and environments. In particular, we consider 8 safety-critical testing scenarios following National Highway Traffic Safety Administration (NHTSA) and develop 4 scenario generation algorithms considering 10 variations for each scenario. Meanwhile, we implement 4 deep reinforcement learning-based AD algorithms with 4 types of input (e.g., bird's-eye view, camera) to perform fair comparisons on SafeBench. We find our generated testing scenarios are indeed more challenging and observe the trade-off between the performance of AD agents under benign and safety-critical testing scenarios. We believe our unified platform SafeBench for large-scale and effective autonomous driving testing will motivate the development of new testing scenario generation and safe AD algorithms. SafeBench is available at https://safebench.github.io. * Equal Contribution Preprint. Under review.

show abstract

“…For a specific operational design domain (ODD), the simulation-based TES conducts billions of miles or scenarios to verify the system performance of automated vehicles [ 9 ]. However, large-scale scenarios are still a serious challenge for the testing efficiency of the simulation-based testing system [ 10 ]. What is more, typical naturalistic driving data (NDD) brings reliable but low-coverage test results, while the ideal scenario space data (ISSD) brings high-coverage but low-confidence test results [ 11 , 12 ].…”

Section: Introductionmentioning

confidence: 99%

Research on the Physics–Intelligence Hybrid Theory Based Dynamic Scenario Library Generation for Automated Vehicles

Zhang

Sun

et al. 2022

Sensors

View full text Add to dashboard Cite

The testing and evaluation system has been the key technology and security with its necessity in the development and deployment of maturing automated vehicles. In this research, the physics–intelligence hybrid theory-based dynamic scenario library generation method is proposed to improve system performance, in particular, the testing efficiency and accuracy for automated vehicles. A general framework of the dynamic scenario library generation is established. Then, the parameterized scenario based on the dimension optimization method is specified to obtain the effective scenario element set. Long-tail functions for performance testing of specific ODD are constructed as optimization boundaries and critical scenario searching methods are proposed based on the node optimization and sample expansion methods for the low-dimensional scenario library generation and the reinforcement learning for the high-dimensional one, respectively. The scenario library generation method is evaluated with the naturalistic driving data (NDD) of the intelligent electric vehicle in the field test. Results show better efficient and accuracy performances compared with the ideal testing library and the NDD, respectively, in both low- and high-dimensional scenarios.

show abstract

A scenario generation pipeline for autonomous vehicle simulators

Cited by 20 publications

References 32 publications

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehicles

Research on the Physics–Intelligence Hybrid Theory Based Dynamic Scenario Library Generation for Automated Vehicles

Contact Info

Product

Resources

About