Assessing the Safety of Environment Perception in Automated Driving Vehicles

Berk, Mario; Schubert, Olaf; Kroll, Hans-Martin; Buschardt, Boris; Štraub, Daniel

doi:10.4271/09-08-01-0004

Cited by 18 publications

(28 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors of Ref. [49] provide a simple approach toward the safety-criticality of perception errors in the existence uncertainty domain. Certain fractions of the binary error types FP and FN are assumed to be safety-critical, where the fractions depend on the error's position of occurrence within the ego vehicle's FOV.…”

Section: Heuristic For the Safety-relevancementioning

confidence: 99%

“…Certain fractions of the binary error types FP and FN are assumed to be safety-critical, where the fractions depend on the error's position of occurrence within the ego vehicle's FOV. For example, perception errors directly in front of the ego vehicle can heuristically be estimated to be more likely to be safety-critical than perception errors occurring farther away or with a lateral offset [49]. The benefit of this approach is that once its numerical values are set, it does not need to consider any downstream driving function for computing safety-critical failure rates.…”

Section: Heuristic For the Safety-relevancementioning

confidence: 99%

“…The above-mentioned fractions of safety-critical perception errors could be determined by means of closed-loop fault injection simulations [49]. However, if the downstream driving function receives a minor update, the perception metrics would have to be recomputed, which would render such a direct approach unpractical for iterative development.…”

Section: Modeling the Perception-control Linkagementioning

confidence: 99%

See 2 more Smart Citations

A Review of Testing Object-Based Environment Perception for Safe Automated Driving

2022

View full text Add to dashboard Cite

Safety assurance of automated driving systems must consider uncertain environment perception. This paper reviews literature addressing how perception testing is realized as part of safety assurance. The paper focuses on testing for verification and validation purposes at the interface between perception and planning, and structures the analysis along the three axes (1) test criteria and metrics, (2) test scenarios, and (3) reference data. Furthermore, the analyzed literature includes related safety standards, safety-independent perception algorithm benchmarking, and sensor modeling. It is found that the realization of safety-oriented perception testing remains an open issue since challenges concerning the three testing axes and their interdependencies currently do not appear to be sufficiently solved.

show abstract

Section: Heuristic For the Safety-relevancementioning

confidence: 99%

Section: Heuristic For the Safety-relevancementioning

confidence: 99%

Section: Modeling the Perception-control Linkagementioning

confidence: 99%

See 1 more Smart Citation

A Review of Testing Object-Based Environment Perception for Safe Automated Driving

2022

View full text Add to dashboard Cite

show abstract

“…The performance of an ADS depends on that of the EV localization, perception, motion planning, and vehicle control module (Berk et al, 2020). Perception and motion planning modules are the most complicated and influencing parts of an ADS.…”

Section: Environment Perception Motion Planning and Their Performance Metricsmentioning

confidence: 99%

“…Self-developed performance metrics are being employed to improve the ADS, using data from limited field deployments. Several other research studies related to the safety of ADS are already published (e.g., Alkhoury, 2017;Every et al, 2017;Fraade-blanar et al, 2018;Nistér et al, 2019;Wood et al, 2019;Berk et al, 2020;Riedmaier et al, 2020;Weng et al, 2020;Wishart et al, 2020;Bansal et al, 2021;Elli et al, 2021;Huang and Kurniawati, 2021;Luiten et al, 2021;Wang et al, 2021). Researchers across the globe are making a considerable effort to quantify the safety of ADS and consequently improve it.…”

Section: Introductionmentioning

confidence: 99%

A Literature Review of Performance Metrics of Automated Driving Systems for On-Road Vehicles

Sharath

Mehran

2021

Front. Future Transp.

View full text Add to dashboard Cite

The article presents a review of recent literature on the performance metrics of Automated Driving Systems (ADS). More specifically, performance indicators of environment perception and motion planning modules are reviewed as they are the most complicated ADS modules. The need for the incorporation of the level of threat an obstacle poses in the performance metrics is described. A methodology to quantify the level of threat of an obstacle is presented in this regard. The approach involves simultaneously considering multiple stimulus parameters (that elicit responses from drivers), thereby not ignoring multivariate interactions. Human-likeness of ADS is a desirable characteristic as ADS share road infrastructure with humans. The described method can be used to develop human-like perception and motion planning modules of ADS. In this regard, performance metrics capable of quantifying human-likeness of ADS are also presented. A comparison of different performance metrics is then summarized. ADS operators have an obligation to report any incident (crash/disengagement) to safety regulating authorities. However, precrash events/states are not being reported. The need for the collection of the precrash scenario is described. A desirable modification to the data reporting/collecting is suggested as a framework. The framework describes the precrash sequences to be reported along with the possible ways of utilizing such a valuable dataset (by the safety regulating authorities) to comprehensively assess (and consequently improve) the safety of ADS. The framework proposes to collect and maintain a repository of precrash sequences. Such a repository can be used to 1) comprehensively learn and model the precrash scenarios, 2) learn the characteristics of precrash scenarios and eventually anticipate them, 3) assess the appropriateness of the different performance metrics in precrash scenarios, 4) synthesize a diverse dataset of precrash scenarios, 5) identify the ideal configuration of sensors and algorithms to enhance safety, and 6) monitor the performance of perception and motion planning modules.

show abstract

Systematic Modeling Approach for Environmental Perception Limitations in Automated Driving

Adee

Gansch

Liggesmeyer

2021

2021 17th European Dependable Computing Conference (EDCC)

View full text Add to dashboard Cite

Highly automated driving (HAD) vehicles are complex systems operating in an open context. Complexity of these systems as well as limitations and insufficiencies in sensing and understanding the open context may result in unsafe and uncertain behavior. The safety critical nature of the HAD vehicles demands to model limitations, insufficiencies and triggering conditions to argue safe behavior.Standardization activities such as ISO/PAS 21448 provide guidelines on the safety of the intended functionality (SOTIF) and focus on the performance limitations and triggering conditions. Although, SOTIF provides a non-exhaustive list of scenario factors that may serve as a starting point to identify and analyze performance limitations and triggering conditions, yet no concrete methodology is provided to model these factors.We propose a novel methodology to model triggering conditions and performance limitations in a scene to assess SOTIF. We utilize Bayesian network (BN) in this regard. The experts provide the BN structure and conditional belief tables are learned using the maximum likelihood estimator. We provide performance limitation maps (PLMs) and conditional performance limitation maps (CPLMs), given a scene. As a case study, we provide PLMs and CPLMs of LIDAR in a defined scene using real world data.

show abstract

Assessing the Safety of Environment Perception in Automated Driving Vehicles

Cited by 18 publications

References 0 publications

A Review of Testing Object-Based Environment Perception for Safe Automated Driving

A Review of Testing Object-Based Environment Perception for Safe Automated Driving

A Literature Review of Performance Metrics of Automated Driving Systems for On-Road Vehicles

Systematic Modeling Approach for Environmental Perception Limitations in Automated Driving

Contact Info

Product

Resources

About