Expert Level Evaluations for Explainable AI (XAI) Methods in the Medical Domain

Muddamsetty, Satya M.; Jahromi, Mohammad N. S.; Moeslund, Thomas B.

doi:10.1007/978-3-030-68796-0_3

Cited by 25 publications

(17 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many of the sources do not make clear the extent to which GPs were involved in the research. In recent explainable artificial intelligence (XAI) efforts to solve various problems in the medical domain, for example, there is a growing need for doctors to be more involved in the development and evaluation of AI diagnostic support tools and systems [ 28 ]. It is not clear, however, whether this same need exists for administrative tasks in general practice, but the current level of GP involvement in administrative tasks appears low.…”

Section: Discussionmentioning

confidence: 99%

Machine learning in general practice: scoping review of administrative task support and automation

et al. 2023

Self Cite

View full text Add to dashboard Cite

Background Artificial intelligence (AI) is increasingly used to support general practice in the early detection of disease and treatment recommendations. However, AI systems aimed at alleviating time-consuming administrative tasks currently appear limited. This scoping review thus aims to summarize the research that has been carried out in methods of machine learning applied to the support and automation of administrative tasks in general practice. Methods Databases covering the fields of health care and engineering sciences (PubMed, Embase, CINAHL with full text, Cochrane Library, Scopus, and IEEE Xplore) were searched. Screening for eligible studies was completed using Covidence, and data was extracted along nine research-based attributes concerning general practice, administrative tasks, and machine learning. The search and screening processes were completed during the period of April to June 2022. Results 1439 records were identified and 1158 were screened for eligibility criteria. A total of 12 studies were included. The extracted attributes indicate that most studies concern various scheduling tasks using supervised machine learning methods with relatively low general practitioner (GP) involvement. Importantly, four studies employed the latest available machine learning methods and the data used frequently varied in terms of setting, type, and availability. Conclusion The limited field of research developing in the application of machine learning to administrative tasks in general practice indicates that there is a great need and high potential for such methods. However, there is currently a lack of research likely due to the unavailability of open-source data and a prioritization of diagnostic-based tasks. Future research would benefit from open-source data, cutting-edge methods of machine learning, and clearly stated GP involvement, so that improved and replicable scientific research can be done.

show abstract

Section: Discussionmentioning

confidence: 99%

Machine learning in general practice: scoping review of administrative task support and automation

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…Samek et al (2021) is one such example, classifying XAI algorithms by the various mathematical approaches of explanation generation: local surrogates, occlusions, gradient‐based techniques, and layer‐wise relevance propagation. XAI taxonomies also are studied for specific research domains, such as medical image analytics (Muddamsetty et al, 2021). Another way to taxonomize XAI is by function and medium (e.g., algorithms, visualization, audio, KGs, and plain text) (e.g., Rawal et al, 2021).…”

Section: Geographic Applications Of Xai Methods: State‐of‐the‐artmentioning

confidence: 99%

The challenges of integrating explainable artificial intelligence into GeoAI

Jin

Sieber

2023

Transactions in GIS

View full text Add to dashboard Cite

Although explainable artificial intelligence (XAI) promises considerable progress in glassboxing deep learning models, there are challenges in applying XAI to geospatial artificial intelligence (GeoAI), specifically geospatial deep neural networks (DNNs). We summarize these as three major challenges, related generally to XAI computation, to GeoAI and geographic data handling, and to geosocial issues. XAI computation includes the difficulty of selecting reference data/models and the shortcomings of attributing explanatory power to gradients, as well as the difficulty in accommodating geographic scale, geovisualization, and underlying geographic data structures. Geosocial challenges encompass the limitations of knowledge scope—semantics and ontologies—in the explanation of GeoAI as well as the lack of integrating non‐technical aspects in XAI, including processes that are not amenable to XAI. We illustrate these issues with a land use classification case study.

show abstract

“…Model explainability is essential for gaining trust and acceptance of AI systems in high-stakes areas, such as healthcare, where reliability and safety are critical [43], [44]. Medical anomaly detection [45], healthcare risk prediction system [46], [47], [48], [49], genetics [50], [51], and healthcare image processing [52], [53], [54] are some of the areas that are moving towards adoption of XAI. Another area is finance, such as AI-based credit score decisions [55], [56] and counterfeit banknotes detection [57].…”

Section: Explainable Artificial Intelligence (Xai)mentioning

confidence: 99%

Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities

et al. 2022

View full text Add to dashboard Cite

The application of Artificial Intelligence (AI) and Machine Learning (ML) to cybersecurity challenges has gained traction in industry and academia, partially as a result of widespread malware attacks on critical systems such as cloud infrastructures and government institutions. Intrusion Detection Systems (IDS), using some forms of AI, have received widespread adoption due to their ability to handle vast amounts of data with a high prediction accuracy. These systems are hosted in the organizational Cyber Security Operation Center (CSoC) as a defense tool to monitor and detect malicious network flow that would otherwise impact the Confidentiality, Integrity, and Availability (CIA). CSoC analysts rely on these systems to make decisions about the detected threats. However, IDSs designed using Deep Learning (DL) techniques are often treated as black box models and do not provide a justification for their predictions. This creates a barrier for CSoC analysts, as they are unable to improve their decisions based on the model's predictions. One solution to this problem is to design explainable IDS (X-IDS). This survey reviews the state-of-the-art in explainable AI (XAI) for IDS, its current challenges, and discusses how these challenges span to the design of an X-IDS. In particular, we discuss black box and white box approaches comprehensively. We also present the tradeoff between these approaches in terms of their performance and ability to produce explanations. Furthermore, we propose a generic architecture that considers human-in-the-loop which can be used as a guideline when designing an X-IDS. Research recommendations are given from three critical viewpoints: the need to define explainability for IDS, the need to create explanations tailored to various stakeholders, and the need to design metrics to evaluate explanations.INDEX TERMS Explainable intrusion detection systems, explainable artificial intelligence, machine learning, deep learning, white box, black box, explainability, cybersecurity.

show abstract

Expert Level Evaluations for Explainable AI (XAI) Methods in the Medical Domain

Cited by 25 publications

References 22 publications

Machine learning in general practice: scoping review of administrative task support and automation

Machine learning in general practice: scoping review of administrative task support and automation

The challenges of integrating explainable artificial intelligence into GeoAI

Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities

Contact Info

Product

Resources

About