AI and Reliability Trends in Safety-Critical Autonomous Systems on Ground and Air

Athavale, Jyotika; Baldovin, Andrea; Graefe, Ralf; Paulitsch, Michael; Rosales, Rafael

doi:10.1109/dsn-w50199.2020.00024

Cited by 28 publications

(12 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides the risk of data misuse, AI systems are vulnerable to both software and hardware faults, that can be extremely harmful for patients. For instance, inadequate training data or wrong design choices can cause abnormal system behavior ( 31 ). These errors can be due to the users rather than to the developers: for instance, an AI system can make wrong (and potentially harmful) decisions if it is not used in the original design context ( 31 ).…”

Section: Current Medico-legal and Ethical Challengesmentioning

confidence: 99%

“…For instance, inadequate training data or wrong design choices can cause abnormal system behavior ( 31 ). These errors can be due to the users rather than to the developers: for instance, an AI system can make wrong (and potentially harmful) decisions if it is not used in the original design context ( 31 ). Moreover, logic, memory or communication components of the devices can be affected by permanent or transient hardware failures (like the transient failures—also called “soft errors”—represented by bit flips due to radiation particles) ( 31 ).…”

Section: Current Medico-legal and Ethical Challengesmentioning

confidence: 99%

See 1 more Smart Citation

Management of Medico-Legal Risks in Digital Health Era: A Scoping Review

et al. 2022

View full text Add to dashboard Cite

Artificial intelligence needs big data to develop reliable predictions. Therefore, storing and processing health data is essential for the new diagnostic and decisional technologies but, at the same time, represents a risk for privacy protection. This scoping review is aimed at underlying the medico-legal and ethical implications of the main artificial intelligence applications to healthcare, also focusing on the issues of the COVID-19 era. Starting from a summary of the United States (US) and European Union (EU) regulatory frameworks, the current medico-legal and ethical challenges are discussed in general terms before focusing on the specific issues regarding informed consent, medical malpractice/cognitive biases, automation and interconnectedness of medical devices, diagnostic algorithms and telemedicine. We aim at underlying that education of physicians on the management of this (new) kind of clinical risks can enhance compliance with regulations and avoid legal risks for the healthcare professionals and institutions.

show abstract

Section: Current Medico-legal and Ethical Challengesmentioning

confidence: 99%

Section: Current Medico-legal and Ethical Challengesmentioning

confidence: 99%

Management of Medico-Legal Risks in Digital Health Era: A Scoping Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…For example, Jenihhin et al (2019) reviewed the challenges of reliability assessment and enhancement in autonomous systems. Athavale et al (2020) discussed the trends in AI reliability in safety-critical autonomous systems on the ground and in the air.…”

Section: The Importance Of Ai Reliabilitymentioning

confidence: 99%

Statistical Perspectives on Reliability of Artificial Intelligence Systems

Hong¹,

Lian²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Artificial intelligence (AI) systems have become increasingly popular in many areas.Nevertheless, AI technologies are still in their developing stages, and many issues need to be addressed. Among those, the reliability of AI systems needs to be demonstrated so that the AI systems can be used with confidence by the general public. In this paper, we provide statistical perspectives on the reliability of AI systems. Different from other considerations, the reliability of AI systems focuses on the time dimension. That is, the system can perform its designed functionality for the intended period. We introduce a so-called "SMART" statistical framework for AI reliability research, which includes five components: Structure of the system, Metrics of reliability, Analysis of failure causes, Reliability assessment, and Test planning. We review traditional methods in reliability data analysis and software reliability, and discuss how those existing methods can be transformed for reliability modeling and assessment of AI systems. We also describe recent developments in modeling and analysis of AI reliability and outline statistical research challenges in this area, including out-of-distribution detection, the effect of the training set, adversarial attacks, model accuracy, and uncertainty quantification, and discuss how those topics can be related to AI reliability, with illustrative examples. Finally, we discuss data collection and test planning for AI reliability assessment and how to improve system designs for higher AI reliability. The paper closes with some concluding remarks.

show abstract

“…Driven by research challenges such as autonomous driving, there is a substantial research effort to define AI solutions for the development of safety-critical systems [3], [4], [8], [9], aligned with the required evolution and definition of new safety standards. This is also of interest in other domains -for example, in industrial domains such as railway interlocking-where AI solutions could also be used to develop safety-critical systems [1], [10], [11].…”

Section: Introductionmentioning

confidence: 99%

Optimization Techniques and Formal Verification for the Software Design of Boolean Algebra Based Safety-Critical Systems

Pérez

Flores

Blum

et al. 2022

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

Artificial intelligence, and the ability to learn optimized solutions that comply with a set of safety rules, could facilitate the human-based design process of safety-critical systems. However, the reconciliation of state-of-the-art artificial intelligence technology with current safety standards and safety engineering processes is a challenge to be addressed. This publication describes a method based on optimization and on formal verification for the design of safety-critical systems that are defined by Boolean algebra. Several diverse optimization techniques and a hybrid of these approaches are used to find an optimized design that considers performance requirements, availability rules and complies with all defined safety rules. Subsequently, this solution is translated into an alternative knowledge representation that can be formally verified and developed in compliance with currently considered safety standards. This method is evaluated with a simplified safety-critical case study.

show abstract

AI and Reliability Trends in Safety-Critical Autonomous Systems on Ground and Air

Cited by 28 publications

References 8 publications

Management of Medico-Legal Risks in Digital Health Era: A Scoping Review

Management of Medico-Legal Risks in Digital Health Era: A Scoping Review

Statistical Perspectives on Reliability of Artificial Intelligence Systems

Optimization Techniques and Formal Verification for the Software Design of Boolean Algebra Based Safety-Critical Systems

Contact Info

Product

Resources

About