Dynamic Assurance Cases: A Pathway to Trusted Autonomy

Asaadi, Erfan; Denney, Ewen; Menzies, Jonathan; Pai, Ganesh; Petroff, Dimo

doi:10.1109/mc.2020.3022030

Cited by 30 publications

(17 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[31]) and retraining methods [6] with the RAM, to form a closed loop of debugging-improving-assessing. Last but not least, we find the idea of dynamic assurance cases [2] may have a great potential for addressing some challenges we currently face in our framework.…”

Section: Discussionmentioning

confidence: 98%

“…during operation, and to redo the cell partition when the estimated r has changed significantly. Such a dynamic way of estimating r can be supported by the concept of dynamic assurance cases [2].…”

Section: Discussion On the Proposed Rammentioning

confidence: 99%

“…If the OP was subject to sudden changes, change-point detectors like [70] should also be paired with the runtime estimator to robustly approximate the OP. Again, such dynamic way of estimating OP can also be supported by dynamic assurance cases [2].…”

Section: Discussion On the Proposed Rammentioning

confidence: 99%

“…In [41], safety arguments that are being widely used for conventional systems-including conformance to standards, proven in use, field testing, simulation, and formal proofs-are recapped for autonomous systems with discussions on the potential pitfalls. Both, [48] and [34], propose utilising continuously updated arguments to monitor the weak points and the effectiveness of their countermeasures, while a similar mechanism is also suggested in our assurance case, e.g., continuously monitor/estimate key parameters of our RAM-all essentially aligns with the idea of dynamic assurance cases [20,2].…”

Section: Related Workmentioning

confidence: 93%

See 3 more Smart Citations

Reliability Assessment and Safety Arguments for Machine Learning Components in System Assurance

Dong¹,

Huang²,

Bharti³

et al. 2021

Preprint

View full text Add to dashboard Cite

The increasing use of Machine Learning (ML) components embedded in autonomous systems -so-called Learning-Enabled Systems (LES) -has resulted in the pressing need to assure their functional safety. As for traditional functional safety, the emerging consensus within both, industry and academia, is to use assurance cases for this purpose. Typically assurance cases support claims of reliability in support of safety, and can be viewed as a structured way of organising arguments and evidence generated from safety analysis and reliability modelling activities. While such assurance activities are traditionally guided by consensus-based standards developed from vast engineering experience, LES pose new challenges in safety-critical application due to the characteristics and design of ML models. In this article, we first present an overall assurance framework for LES with an emphasis on quantitative aspects, e.g., breaking down system-level safety targets to component-level requirements and supporting claims stated in reliability metrics. We then introduce a novel model-agnostic Reliability Assessment Model (RAM) for ML classifiers that utilises the operational profile and robustness verification evidence. We discuss the model assumptions and the inherent challenges of assessing ML reliability uncovered by our RAM and propose practical solutions. Probabilistic safety arguments at the lower ML component-level are also developed based on the RAM. Finally, to evaluate and demonstrate our methods, we not only conduct experiments on synthetic/benchmark datasets but also demonstrate the scope of our methods with a comprehensive case study on Autonomous Underwater Vehicles in simulation.

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Discussion On the Proposed Rammentioning

confidence: 99%

Section: Discussion On the Proposed Rammentioning

confidence: 99%

Section: Related Workmentioning

confidence: 93%

See 2 more Smart Citations

Reliability Assessment and Safety Arguments for Machine Learning Components in System Assurance

Dong¹,

Huang²,

Bharti³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…to regulators, key decisions about the choice of the ML model and quality and suitability of the data sets. Post-deployment, local XAI methods could help to implement a highly dynamic assurance case [ 78 , 79 ] where the critical predictions made by an ML-based system could be used to update the assumptions about, and confidence in, the system deployed compared with the assessment made pre-deployment.…”

Section: Discussionmentioning

confidence: 99%

Artificial intelligence explainability: the technical and ethical dimensions

McDermid

Jia

Porter

et al. 2021

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

In recent years, several new technical methods have been developed to make AI-models more transparent and interpretable. These techniques are often referred to collectively as ‘AI explainability’ or ‘XAI’ methods. This paper presents an overview of XAI methods, and links them to stakeholder purposes for seeking an explanation. Because the underlying stakeholder purposes are broadly ethical in nature, we see this analysis as a contribution towards bringing together the technical and ethical dimensions of XAI. We emphasize that use of XAI methods must be linked to explanations of human decisions made during the development life cycle. Situated within that wider accountability framework, our analysis may offer a helpful starting point for designers, safety engineers, service providers and regulators who need to make practical judgements about which XAI methods to employ or to require. This article is part of the theme issue ‘Towards symbiotic autonomous systems’.

show abstract