Safety Assurance of Machine Learning for Chassis Control Functions

Burton, Simon; Kurzidem, Iwo; Schwaiger, Adrian; Schleiss, Philipp; Unterreiner, Michael; Graeber, Torben; Becker, Philipp

doi:10.1007/978-3-030-83903-1_10

Cited by 10 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, considering a road surface's traction to be very slippery even in warm and dry weather conditions would lead to an overly pessimistic choice of safety margins. By introducing multiple µODDs and being able to differentiate between different types of road surface conditions during run-time, it is however possible to argue the safety of more optimistic safety margin in case of nonslippery road conditions, ultimately increasing the system's overall utility [14].…”

Section: From Acceptable Risk To DL Performance a Performance Targetmentioning

confidence: 99%

Towards the Quantitative Verification of Deep Learning for Safe Perception

Schleiss

Hagiwara

Kurzidem

et al. 2022

2022 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)

Self Cite

View full text Add to dashboard Cite

Deep learning (DL) is seen as an inevitable building block for perceiving the environment with sufficient detail and accuracy as required by automated driving functions. Despite this, its black-box nature and the therewith intertwined unpredictability still hinders its use in safety-critical systems. As such, this work addresses the problem of making this seemingly unpredictable nature measurable by providing a risk-based verification strategy, such as required by ISO 21448. In detail, a method is developed to break down acceptable risk into quantitative performance targets of individual DL-based components along the perception architecture. To verify these targets, the DL input space is split into areas according to the dimensions of a finegrained operational design domain (µODD). As it is not feasible to reach full test coverage, the strategy suggests to distribute test efforts across these areas according to the associated risk. Moreover, the testing approach provides answers with respect to how much test coverage and confidence in the test result is required and how these figures relate to safety integrity levels (SILs).

show abstract

Section: From Acceptable Risk To DL Performance a Performance Targetmentioning

confidence: 99%

Towards the Quantitative Verification of Deep Learning for Safe Perception

Schleiss

Hagiwara

Kurzidem

et al. 2022

2022 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore it stresses the importance that the safety considerations are meaningful only when scoped within the wider system and operational context. Such an iterative approach is further developed in Burton et al (2021) where a safety assurance argument for a simple ML-based function is discussed. The simplicity of the function and choice of ML technology (an adaptive approach to generalized learning vector quantization Sato and Yamada, 1995) allowed the authors to develop a convincing and comprehensive case by exploiting properties of the environment and model that could be determined with high certainty.…”

Section: Introductionmentioning

confidence: 99%

“…Figure3describes the structure of an assurance argument for a safety-relevant function implemented using supervised ML. The structure is based on a synthesis of previous work in this area in both structuring the assurance argument and defining associated evidences, includingBurton et al (2017),Ashmore et al (2021),Burton et al (2021),Hawkins et al (2021) andHouben et al (2022). This structure is used to reason about which manifestations of uncertainty are addressed by such arguments, whilst an evaluation of the effectiveness of this structure is provided in Section 6.…”

mentioning

confidence: 99%

Addressing uncertainty in the safety assurance of machine-learning

Burton

Herd

2023

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

There is increasing interest in the application of machine learning (ML) technologies to safety-critical cyber-physical systems, with the promise of increased levels of autonomy due to their potential for solving complex perception and planning tasks. However, demonstrating the safety of ML is seen as one of the most challenging hurdles to their widespread deployment for such applications. In this paper we explore the factors which make the safety assurance of ML such a challenging task. In particular we address the impact of uncertainty on the confidence in ML safety assurance arguments. We show how this uncertainty is related to complexity in the ML models as well as the inherent complexity of the tasks that they are designed to implement. Based on definitions of uncertainty as well as an exemplary assurance argument structure, we examine typical weaknesses in the argument and how these can be addressed. The analysis combines an understanding of causes of insufficiencies in ML models with a systematic analysis of the types of asserted context, asserted evidence and asserted inference within the assurance argument. This leads to a systematic identification of requirements on the assurance argument structure as well as supporting evidence. We conclude that a combination of qualitative arguments combined with quantitative evidence are required to build a robust argument for safety-related properties of ML functions that is continuously refined to reduce residual and emerging uncertainties in the arguments after the function has been deployed into the target environment.

show abstract

“…With recent improvements in various technologies, such as sensors, vehicle-to-everything (V2X) communication and machine learning (ML), connected and automated vehicles (CAVs) will sooner than later enter the consumer market. However, safety assurance of CAVs is still a much debated topic and a vivid research area [1], [2]. As CAVs are deployed in a complex and open environment, any incorrect interpretation of its surroundings can lead to unsafe interactions within it.…”

Section: Introductionmentioning

confidence: 99%

SafeSens - Uncertainty Quantification of Complex Perception Systems

Kurzidem,

Burton,

Schleiss

2023

2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

Safety testing and validation of complex autonomous systems requires a comprehensive and reliable analysis of performance and uncertainty. Especially uncertainty quantification plays a vital part in perception systems operating in open context environments that are neither foreseeable nor deterministic. Therefore, safety assurance based on field tests or corner cases alone is not a feasible option as effort and potential risks are high. Simulations offer a way out. They allow, for example, simulation of potentially hazardous situations, without any real danger, by systematically computing a variety of different (input) parameters quickly. In order to do so, simulations need accurate models to represent the complex system and in particular include uncertainty as inherent property to accurately reflect the interdependence between system components and the environment. We present an approach to creating perception architectures via suitable meta-models to enable a holistic safety analysis to quantify the uncertainties within the system. The models include aleatoric or epistemic uncertainty, dependent on the nature of the approximated component. A showcase of the proposed method highlights, how validation under uncertainty can be used for a camera-based object detection.

show abstract

Safety Assurance of Machine Learning for Chassis Control Functions

Cited by 10 publications

References 9 publications

Towards the Quantitative Verification of Deep Learning for Safe Perception

Towards the Quantitative Verification of Deep Learning for Safe Perception

Addressing uncertainty in the safety assurance of machine-learning

SafeSens - Uncertainty Quantification of Complex Perception Systems

Contact Info

Product

Resources

About