Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)

Adadi, Amina; Berrada, Mohammed

doi:10.1109/access.2018.2870052

Cited by 3,802 publications

(2,712 citation statements)

References 59 publications

Supporting

Mentioning

2,653

Contrasting

Unclassified

Order By: Relevance

“…It achieves better prediction, but may not contribute in understanding of the underlying phenomenon. Recently, interpretable machine learning models (explainable AI) are of broad interest [23,24]. In future work, it would be fruitful to extend our method by incorporating some of these ideas for better interpretability.…”

Section: Resultsmentioning

confidence: 99%

Data integration for accelerated materials design via preference learning

Sun¹,

Hou²,

Sumita³

et al. 2020

New J. Phys.

View full text Add to dashboard Cite

Machine learning applications in materials science are often hampered by shortage of experimental data. Integration with external datasets from past experiments is a viable way to solve the problem. But complex calibration is often necessary to use the data obtained under different conditions. In this paper, we present a novel calibration-free strategy to enhance the performance of Bayesian optimization with preference learning. The entire learning process is solely based on pairwise comparison of quantities (i.e., higher or lower) in the same dataset, and experimental design can be done without comparing quantities in different datasets. We demonstrate that Bayesian optimization is significantly enhanced via data integration for organic molecules and inorganic solid-state materials. Our method increases the chance that public datasets are reused and may encourage data sharing in various fields of physics.

show abstract

Section: Resultsmentioning

confidence: 99%

Data integration for accelerated materials design via preference learning

Sun¹,

Hou²,

Sumita³

et al. 2020

New J. Phys.

View full text Add to dashboard Cite

show abstract

“…Various interpretable ML and post-hoc analysis methods have been developed. Adadi and Berrada [1] provided a comprehensive survey of interpretation methods. Here, we describe only the methods related to tree-based or deep-learning based models.…”

Section: Interpretation Methods For Tree and Deep-learning Based Modelsmentioning

confidence: 99%

“…Even though this NN model had a high accuracy rate, the model determined that pneumonia patients with asthma should not be admitted, thinking that these patients have a lower risk of dying. This dubious prediction is caused by the fact that these severe patients had been aggressively treated in intensive care units and, as a result, they survived at a high rate [1,3]. As shown in this example, the decision relied on an ML model might cause critical harm to patients.…”

Section: Introductionmentioning

confidence: 99%

“…To address this problem, data scientists have developed several XAI methods to provide interpretable information of ML models [1]. For example, these methods can show which features of a patient's record have a strong influence on the decision from an ML model (called local feature contributions [34]).…”

Section: Introductionmentioning

confidence: 99%

“…We use model-agnostic interpretation methods [1] for systematic comparisons, in conjunction with several unsupervised methods. Specifically, across the models, we visualize the similarities of local feature contributions for each patient with a dimensionality reduction method.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A visual analytics system for multi-model comparison on clinical data predictions

Fujiwara

Choi

et al. 2020

Visual Informatics

View full text Add to dashboard Cite

There is a growing trend of applying machine learning methods to medical datasets in order to predict patients' future status. Although some of these methods achieve high performance, challenges still exist in comparing and evaluating different models through their interpretable information. Such analytics can help clinicians improve evidence-based medical decision making. In this work, we develop a visual analytics system that compares multiple models' prediction criteria and evaluates their consistency. With our system, users can generate knowledge on different models' inner criteria and how confidently we can rely on each model's prediction for a certain patient. Through a case study of a publicly available clinical dataset, we demonstrate the effectiveness of our visual analytics system to assist clinicians and researchers in comparing and quantitatively evaluating different machine learning methods.

show abstract

Diagnosing Solid Lesions in the Pancreas With Multimodal Artificial Intelligence

Cui,

Zhao,

Xiong

et al. 2024

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceDiagnosing solid lesions in the pancreas via endoscopic ultrasonographic (EUS) images is challenging. Artificial intelligence (AI) has the potential to help with such diagnosis, but existing AI models focus solely on a single modality.ObjectiveTo advance the clinical diagnosis of solid lesions in the pancreas through developing a multimodal AI model integrating both clinical information and EUS images.Design, Setting, and ParticipantsIn this randomized crossover trial conducted from January 1 to June 30, 2023, from 4 centers across China, 12 endoscopists of varying levels of expertise were randomly assigned to diagnose solid lesions in the pancreas with or without AI assistance. Endoscopic ultrasonographic images and clinical information of 439 patients from 1 institution who had solid lesions in the pancreas between January 1, 2014, and December 31, 2022, were collected to train and validate the joint-AI model, while 189 patients from 3 external institutions were used to evaluate the robustness and generalizability of the model.InterventionConventional or AI-assisted diagnosis of solid lesions in the pancreas.Main Outcomes and MeasuresIn the retrospective dataset, the performance of the joint-AI model was evaluated internally and externally. In the prospective dataset, diagnostic performance of the endoscopists with or without the AI assistance was compared.ResultsThe retrospective dataset included 628 patients (400 men [63.7%]; mean [SD] age, 57.7 [27.4] years) who underwent EUS procedures. A total of 130 patients (81 men [62.3%]; mean [SD] age, 58.4 [11.7] years) were prospectively recruited for the crossover trial. The area under the curve of the joint-AI model ranged from 0.996 (95% CI, 0.993-0.998) in the internal test dataset to 0.955 (95% CI, 0.940-0.968), 0.924 (95% CI, 0.888-0.955), and 0.976 (95% CI, 0.942-0.995) in the 3 external test datasets, respectively. The diagnostic accuracy of novice endoscopists was significantly enhanced with AI assistance (0.69 [95% CI, 0.61-0.76] vs 0.90 [95% CI, 0.83-0.94]; P &lt; .001), and the supplementary interpretability information alleviated the skepticism of the experienced endoscopists.Conclusions and RelevanceIn this randomized crossover trial of diagnosing solid lesions in the pancreas with or without AI assistance, the joint-AI model demonstrated positive human-AI interaction, which suggested its potential to facilitate a clinical diagnosis. Nevertheless, future randomized clinical trials are warranted.Trial RegistrationClinicalTrials.gov Identifier: NCT05476978

show abstract

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)

Cited by 3,802 publications

References 59 publications

Data integration for accelerated materials design via preference learning

Data integration for accelerated materials design via preference learning

A visual analytics system for multi-model comparison on clinical data predictions

Diagnosing Solid Lesions in the Pancreas With Multimodal Artificial Intelligence

Contact Info

Product

Resources

About