Explainable Artificial Intelligence for Human Decision Support System in the Medical Domain

Abstract

In this paper, we present the potential of Explainable Artificial Intelligence methods for decision support in medical image analysis scenarios. Using three types of explainable methods applied to the same medical image data set, we aimed to improve the comprehensibility of the decisions provided by the Convolutional Neural Network (CNN). In vivo gastral images obtained by a video capsule endoscopy (VCE) were the subject of visual explanations, with the goal of increasing health professionals’ trust in black-box predictions. We implemented two post hoc interpretable machine learning methods, called Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), and an alternative explanation approach, the Contextual Importance and Utility (CIU) method. The produced explanations were assessed by human evaluation. We conducted three user studies based on explanations provided by LIME, SHAP and CIU. Users from different non-medical backgrounds carried out a series of tests in a web-based survey setting and stated their experience and understanding of the given explanations. Three user groups (n = 20, 20, 20) with three distinct forms of explanations were quantitatively analyzed. We found that, as hypothesized, the CIU-explainable method performed better than both LIME and SHAP methods in terms of improving support for human decision-making and being more transparent and thus understandable to users. Additionally, CIU outperformed LIME and SHAP by generating explanations more rapidly. Our findings suggest that there are notable differences in human decision-making between various explanation support settings. In line with that, we present three potential explainable methods that, with future improvements in implementation, can be generalized to different medical data sets and can provide effective decision support to medical experts.

Towards Machine Learning-Aided Lung Cancer Clinical Routines: Approaches and Open Challenges

“…However, DL models are considered the least interpretable machine learning models due to the inherent mathematical complexity; thus, not providing a reasoning for the prediction and, consequently, decreasing the trust in these models [ 138 ]. When utilizing these black-box models in the medical domain, it is critical to have systems that are trustworthy and reliable to the clinicians, therefore raising the need to make these approaches more transparent and understandable to humans [ 139 ].…”

“…These are off-the-shelf agnostic methods that can be found in libraries, such as PyTorch Captum [ 142 ]. This post-model approach was implemented by Knapič et al [ 139 ], where two popular post-hoc methods, local interpretable model-agnostic explanations (LIME), and SHAPley Additive exPlanations (SHAPs) were compared in terms of understandability for humans in the predictive model with the same medical image dataset.…”

Contextual Importance and Utility: aTheoretical Foundation

“…The CIU Github site https://github.com/ KaryFramling/ciu provides executable examples at least for the well-known benchmark data sets Iris, Boston, Heart Disease, UCI Cars, Diamonds, Titanic and Adult and several different machine learning models. CIU is also implemented for image explanations as reported in [5,7]. The source code used in this paper is published at https://github.com/KaryFramling/AJCAI_2021.…”

Artificial Intelligence Analysis of Gene Expression Predicted the Overall Survival of Mantle Cell Lymphoma and a Large Pan-Cancer Series

“…Neural networks are a favored analytical method for numerous predictive data mining applications because of their power, adaptability, and ease of usage. Predictive neural networks are specially valuable in applications where the underlying process is complex [ 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], such as biological systems [ 44 ]. Both the multilayer perceptron (MLP) and radial basis function (RBF) network have a feedforward architecture, because the connections in the network flow forward the input layer (predictors) to the output layer (responses).…”