A Survey on XAI for Cyber Physical Systems in Medicine

Alimonda, Nicola; Guidotto, Luca; Malandri, Lorenzo; Mercorio, Fabio; Mezzanzanica, Mario; Tosi, Giovanni

doi:10.1109/metroxraine54828.2022.9967673

Cited by 3 publications

(21 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Artificial intelligence affects who we become friends with, how we care for our children and elders, our purchasing and hiring decisions, what we like, our opinions, and the newsfeeds we are shown [12], [26]. AI is used in high-stakes applications affecting human life, including determining credit approval, operation of automated defense machines, in biomedical fields, and in cybersecurity [7], [16]. However, most of these AI applications are not explainable.…”

Section: B Ai Applications and The Black Box Problemmentioning

confidence: 99%

“…XAI models must meet the requirements of demonstrating fidelity as well as being comprehensible, transparent, explainable, understandable, and accurate [7], [34].…”

Section: Definition and Requirements Of Xai Modelsmentioning

confidence: 99%

“…• Comprehensibility or interpretability is defined as "the capacity of a learning system to express its learned information in a human-comprehensible manner" [7], [34].…”

Section: Definition and Requirements Of Xai Modelsmentioning

confidence: 99%

“…These descriptions should provide a clear connection between qualitative and quantitative concepts [34]. • Accuracy: XAI models must demonstrate the ability to correctly predict unseen instances [7]. This can be quantified with evaluation measures such as accuracy, F1 score, and others.…”

Section: Definition and Requirements Of Xai Modelsmentioning

confidence: 99%

“…Few articles exist in the literature on the topic of XAI and general CPS. The great majority of the available articles focus on a specific topic such as XAI for medical CPS [7], XAI applied to industrial CPS [8], or XAI used in CPS security [9], [10]. There is a need for quantitative analysis of the numerous benefits of using XAI with CPS, and there is a lack of standardized methods of quantifying the understandability and performance of XAI.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Explainable AI for Cyber-Physical Systems: Issues and Challenges

Hoenig,

Roy,

Acquaah

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Artificial intelligence and cyber-physical systems (CPS) are two of the key technologies of the future that are enabling major global shifts. However, most of the current implementations of AI in CPS are not explainable, which creates serious problems in ethical, legal, regulatory, and other domains. Therefore, it is necessary for explainable artificial intelligence (XAI) to be integrated with cyber-physical systems to meet the vital needs for control, fairness, accountability, safety, cyber-resilience, and cybersecurity. The goal of this review is to demonstrate the need, benefits, challenges, and implementation of XAI for CPS. We review the existing literature about XAI and CPS, discuss the current state of the art, examine applications in different domains, and make recommendations for future research directions. To the best of our knowledge, this is the first peer-reviewed academic article to provide a comprehensive review of general XAI for CPS. We also contribute new research ideas including development of multisensory explanations and outputs for these systems, application of XAI to CPS to decrease occupational burnout and increase employee engagement, and enumeration of the multidisciplinary goals and benefits of XAI as applied to cyber-physical systems.

show abstract

Section: B Ai Applications and The Black Box Problemmentioning

confidence: 99%

“…XAI models must meet the requirements of demonstrating fidelity as well as being comprehensible, transparent, explainable, understandable, and accurate [7], [34].…”

Section: Definition and Requirements Of Xai Modelsmentioning

confidence: 99%

“…• Comprehensibility or interpretability is defined as "the capacity of a learning system to express its learned information in a human-comprehensible manner" [7], [34].…”

Section: Definition and Requirements Of Xai Modelsmentioning

confidence: 99%

Section: Definition and Requirements Of Xai Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Explainable AI for Cyber-Physical Systems: Issues and Challenges

Hoenig,

Roy,

Acquaah

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

Explainable Machine Learning in Critical Decision Systems: Ensuring Safe Application and Correctness

Wiggerthale,

Reich

2024

View full text Add to dashboard Cite

Machine learning (ML) is increasingly used to support or automate decision processes in critical decision systems such as self driving cars or systems for medical diagnosis. These systems require decisions in which human lives are at stake and the decisions should therefore be well founded and very reliable. This need for reliability contrasts with the black-box nature of many ML models, making it difficult to ensure that they always behave as intended. In face of the high stakes involved, the resulting uncertainty is a significant challenge. Explainable artificial intelligence (XAI) addresses the issue by making black-box models more interpretable, often to increase user trust. However, many current XAI applications focus more on transparency and usability than on enhancing safety of ML applications. In this work, we therefore conduct a systematic literature review to examine how XAI can be leveraged to increase safety of ML applications in critical decision systems. We strive to find out for what purposes XAI is currently used in critical decision systems, what are the most common XAI techniques in critical decision systems and how XAI can be harnessed to increase safety of ML applications in critical decision systems. Using the SPAR-4-SLR protocol, we are able to answer these questions and provide a foundational resource for researchers and practitioners seeking to mitigate risks of ML applications. Essentially, we identify promising approaches of XAI which go beyond increasing trust to actively ensure correctness of decisions. Our findings propose a three-layered framework to enhance safety of ML in critical decision systems by means of XAI. The approach consists of Reliability, Validation and Verification. Furthermore, we point out gaps in research and propose future directions of XAI research for enhancing safety of ML applications in critical decision systems.

show abstract

Exploring Local Explanation of Practical Industrial AI Applications: A Systematic Literature Review

Prihatno

Oktian

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

In recent years, numerous explainable artificial intelligence (XAI) use cases have been developed, to solve numerous real problems in industrial applications while maintaining the explainability level of the used artificial intelligence (AI) models to judge their quality and potentially hold the models accountable if they become corrupted. Therefore, understanding the state-of-the-art methods, pointing out recent issues, and deriving future directions are important to drive XAI research efficiently. This paper presents a systematic literature review of local explanation techniques and their practical applications in various industrial sectors. We first establish the need for XAI in response to opaque AI models and survey different local explanation methods for industrial AI applications. The number of studies is then examined with several factors, including industry sectors, AI models, data types, and XAI-based usage and purpose. We also look at the advantages and disadvantages of local explanation methods and how well they work in practical settings. The difficulties of using local explanation techniques are also covered, including computing complexity and the trade-off between precision and interpretability. Our findings demonstrate that local explanation techniques can boost industrial AI models’ transparency and interpretability and give insightful information about them. The efficiency of these procedures must be improved, and ethical concerns about their application must be resolved. This paper contributes to the increasing knowledge of local explanation strategies and offers guidance to academics and industry professionals who want to use these methods in practical settings.

show abstract

A Survey on XAI for Cyber Physical Systems in Medicine

Cited by 3 publications

References 30 publications

Explainable AI for Cyber-Physical Systems: Issues and Challenges

Explainable AI for Cyber-Physical Systems: Issues and Challenges

Explainable Machine Learning in Critical Decision Systems: Ensuring Safe Application and Correctness

Exploring Local Explanation of Practical Industrial AI Applications: A Systematic Literature Review

Contact Info

Product

Resources

About