A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Ding, Wenhao; Xu, Chejian; Arief, Mansur; Hu, Lin; Li, Bo; Zhao, Ding

doi:10.48550/arxiv.2202.02215

Cited by 3 publications

(4 citation statements)

References 134 publications

(155 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The scenarios faced by UAS in reality are relatively complex, with many error-caused factors, and the parameter values of the factor are continuous scalars. [9]Assumed the distribution of scenario x is parameterized with , the error-caused scenario can be represented by formula (4).…”

Section: Distribution Of Error-caused Scenariosmentioning

confidence: 99%

See 1 more Smart Citation

A Method for Generating Error-caused Scenarios

Du,

Zhu,

Bao

et al. 2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

In recent years, with the development of artificial intelligence, Unmanned Autonomous Systems (UAS) have also developed rapidly. It is necessary to conduct sufficient testing and evaluation of UAS before using it. This paper proposes a method for generating error-caused scenarios that lead to incorrect behavior in UAS, and how to evaluate the level of incorrect behavior in UAS caused by the scenario. Based on the method proposed in this paper, error-caused scenarios can be generated in the simulation platform, and the dataset constructed from these scenarios can be input into the target detection model to verify the effectiveness of error-caused scenarios. Our experiments have shown that the scenarios generated by the method can easily lead to errors in UAS, and the method can play an important role in the field of model-based UAS testing and validation.

show abstract

Section: Distribution Of Error-caused Scenariosmentioning

confidence: 99%

“…The normalized value of the error-caused factors in the scenarios represents the error-caused level of the scenario, and the measurement formula is shown in formula (9).…”

Section: Measurement Of Error-caused Level In Scenariosmentioning

confidence: 99%

A Method for Generating Error-caused Scenarios

Du,

Zhu,

Bao

et al. 2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…The second limitation is that most of the benchmarks use pre-defined tasks and parameters set by the creators, which lacks diversity and may be subject to human biases. Last but not least, although tasks are usually randomly sampled, the distribution rarely triggers critical events with catastrophic consequences, resulting in an underestimation of risks and very slow convergence of the results [40]. Some recent benchmarks [199] use realistic 3D simulators to construct real-world scenarios and use accelerated evaluation methods [10,204] to emphasize the rare safety-critical cases.…”

Section: How To Design Evaluation Platforms For Trustworthy Rl?mentioning

confidence: 99%

Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability

Xu¹,

Liu²,

Huang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

A trustworthy reinforcement learning algorithm should be competent in solving challenging real-world problems, including robustly handling uncertainties, satisfying safety constraints to avoid catastrophic failures, and generalizing to unseen scenarios during deployments. This study aims to overview these main perspectives of trustworthy reinforcement learning considering its intrinsic vulnerabilities on robustness, safety, and generalizability. In particular, we give rigorous formulations, categorize corresponding methodologies, and discuss benchmarks for each perspective. Moreover, we provide an outlook section to spur promising future directions with a brief discussion on extrinsic vulnerabilities considering human feedback. We hope this survey could bring together separate threads of studies together in a unified framework and promote the trustworthiness of reinforcement learning. CCS Concepts: • Computing methodologies → Reinforcement learning; Markov decision processes; • Security and privacy → Social aspects of security and privacy; • Computer systems organization → Robotics; • Hardware → Safety critical systems.

show abstract

“…The third kinds do not optimize the likelihood but use the game theory to find an equilibrium point between a generative model and a discriminator. A good review is provided in [43].…”

Section: Constructing D N1 With Dangerous Scenario Synthesismentioning

confidence: 99%

Test Against High-Dimensional Uncertainties: Accelerated Evaluation of Autonomous Vehicles with Deep Importance Sampling

Arief¹,

Cen²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Evaluating the performance of autonomous vehicles (AV) and their complex subsystems to high precision under naturalistic circumstances remains a challenge, especially when the failure or dangerous cases are rare. Rarity does not only require an enormous sample size for a naive method to achieve high confidence estimation, but it also causes dangerous underestimation of the true failure rate and it is extremely hard to detect. Meanwhile, the state-of-the-art approach that comes with a correctness guarantee can only compute an upper bound for the failure rate under certain conditions, which could limit its practical uses. In this work, we present Deep Importance Sampling (Deep IS) framework that utilizes a deep neural network to obtain an efficient IS that is on-par with the state-of-the-art, capable of reducing the required sample size 43 times smaller than the naive sampling method to achieve 10% relative error and while producing an estimate that is much less conservative. Our high-dimensional experiment estimating the misclassification rate of one of the state-of-the-art traffic sign classifiers further reveals that this efficiency still holds true even when the target is very small, achieving over 600 times efficiency boost. This highlights the potential of Deep IS in providing a precise estimate even against highdimensional uncertainties.

show abstract

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective

Cited by 3 publications

References 134 publications

A Method for Generating Error-caused Scenarios

A Method for Generating Error-caused Scenarios

Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability

Test Against High-Dimensional Uncertainties: Accelerated Evaluation of Autonomous Vehicles with Deep Importance Sampling

Contact Info

Product

Resources

About