2021
DOI: 10.1613/jair.1.12716
|View full text |Cite
|
Sign up to set email alerts
|

A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems

Abstract: Autonomous cyber-physical systems (CPS) can improve safety and efficiency for safety-critical applications, but require rigorous testing before deployment. The complexity of these systems often precludes the use of formal verification and real-world testing can be too dangerous during development. Therefore, simulation-based techniques have been developed that treat the system under test as a black box operating in a simulated environment. Safety validation tasks include finding disturbances in the environment… Show more

Help me understand this report
View preprint versions

Search citation statements

Order By: Relevance

Paper Sections

Select...
3
1
1

Citation Types

0
42
0

Year Published

2021
2021
2024
2024

Publication Types

Select...
5
3
1

Relationship

1
8

Authors

Journals

citations
Cited by 76 publications
(47 citation statements)
references
References 101 publications
(228 reference statements)
0
42
0
Order By: Relevance
“…Sampling can be used to assess the quality of a neural network; neural networks are typically evaluated on a test set after training (Goodfellow et al, 2016). Statistical approaches to verification (Corso et al, 2021) have been used for a wide variety of applications such as autonomous driving (Huang et al, 2018) and aviation (Zakrzewski, 2004). Zakrzewski (2004) investigates the number of samples required to estimate the failure probability with high confidence, considering Bayesian and non-Bayesian settings.…”
Section: Related Workmentioning
confidence: 99%
See 1 more Smart Citation
“…Sampling can be used to assess the quality of a neural network; neural networks are typically evaluated on a test set after training (Goodfellow et al, 2016). Statistical approaches to verification (Corso et al, 2021) have been used for a wide variety of applications such as autonomous driving (Huang et al, 2018) and aviation (Zakrzewski, 2004). Zakrzewski (2004) investigates the number of samples required to estimate the failure probability with high confidence, considering Bayesian and non-Bayesian settings.…”
Section: Related Workmentioning
confidence: 99%
“…If sampling is performed using a distribution that is slightly wrong, the claims of safety assurance fall apart. Consequently, other methods obtain guarantees using sampling coverage metrics over the input space, which again scale exponentially with the dimension of the input domain (Corso et al, 2021). Given the difficulties of sampling-based safety guarantees, we believe formal guarantees which hold over the entire input domain are more compelling.…”
Section: Related Workmentioning
confidence: 99%
“…In contrast, we focus on a more specific topic, scenario-based ADS testing in high-fidelity simulation in much more profound depth. [141] reviews widely used algorithms for black-box safety validation. It focuses on presenting the general algorithms, while we present the algorithms along with other components (e.g., simulators, systems, scenario parameters, and testing objectives) in the context of ADS testing.…”
Section: Related Workmentioning
confidence: 99%
“…But as simulation models are often complex and safety-critical states rare, creating interesting and critical test scenarios becomes a challenge [24]. Often, one uses algorithms for search, optimization, or machine-learning to find simulation conditions that expose hazards of failures [25]. Different heuristics (e.g.…”
Section: Simulation-based Safety Testingmentioning
confidence: 99%