Explainable Artificial Intelligence for Human Decision-Support System in Medical Domain

Abstract

In the present paper we present the potential of Explainable Artificial Intelligence methods for decision-support in medical image analysis scenarios. With three types of explainable methods applied to the same medical image data set our aim was to improve the comprehensibility of the decisions provided by the Convolutional Neural Network (CNN). The visual explanations were provided on in-vivo gastral images obtained from a Video capsule endoscopy (VCE), with the goal of increasing the health professionals' trust in the black box predictions. We implemented two post-hoc interpretable machine learning methods LIME and SHAP and the alternative explanation approach CIU, centered on the Contextual Value and Utility (CIU). The produced explanations were evaluated using human evaluation. We conducted three user studies based on the explanations provided by LIME, SHAP and CIU. Users from different non-medical backgrounds carried out a series of tests in the web-based survey setting and stated their experience and understanding of the given explanations. Three user groups (n=20, 20, 20) with three distinct forms of explanations were quantitatively analyzed. We have found that, as hypothesized, the CIU explainable method performed better than both LIME and SHAP methods in terms of increasing support for human decision-making as well as being more transparent and thus understandable to users. Additionally, CIU outperformed LIME and SHAP by generating explanations more rapidly. Our findings suggest that there are notable differences in human decisionmaking between various explanation support settings. In line with that, we present three potential explainable methods that can with future improvements in implementation be generalized on different medical data sets and can provide great decision-support for medical experts.

Perceive, Understand & Predict - Empirical Indication for Facets in Subjective Information Processing Awareness

“…5.1 Use SIPA to evaluate different explanation approaches [24] compared different explanatory methods (e.g., Lime, SHAPE, and CUI) and found that they have different effects on experienced transparency, understandability, or satisfaction. Overall, several studies indicate a close connection between the function of an explanation and the situation in which it is given, e.g., [5].…”

Towards Machine Learning-Aided Lung Cancer Clinical Routines: Approaches and Open Challenges

“…However, DL models are considered the least interpretable machine learning models due to the inherent mathematical complexity; thus, not providing a reasoning for the prediction and, consequently, decreasing the trust in these models [138]. When utilizing these black-box models in the medical domain, it is critical to have systems that are trustworthy and reliable to the clinicians, therefore raising the need to make these approaches more transparent and understandable to humans [139].…”

“…These are off-the-shelf agnostic methods that can be found in libraries, such as PyTorch Captum [142]. This post-model approach was implemented by Knapič et al [139], where two popular post-hoc methods, local interpretable model-agnostic explanations (LIME), and SHAPley Additive exPlanations (SHAPs) were compared in terms of understandability for humans in the predictive model with the same medical image dataset.…”

Offshore Oil Slick Detection: From Photo-Interpreter to Explainable Multi-Modal Deep Learning Models Using SAR Images and Contextual Data

“…They provide results demonstrating the expressiveness of SHAP values in terms of discrimination ability between different output classes and better alignment with human intuition compared to many other existing methods [62]. Several works have adopted the SHAP method for image classification or object detection [67,68].…”