Explainable deep learning in healthcare: A methodological survey from an attribution view

Abstract

The increasing availability of large collections of electronic health record (EHR) data and unprecedented technical advances in deep learning (DL) have sparked a surge of research interest in developing DL based clinical decision support systems for diagnosis, prognosis, and treatment. Despite the recognition of the value of deep learning in healthcare, impediments to further adoption in real healthcare settings remain due to the black-box nature of DL. Therefore, there is an emerging need for interpretable DL, which allows end users to evaluate the model decision making to know whether to accept or reject predictions and recommendations before an action is taken. In this review, we focus on the interpretability of the DL models in healthcare. We start by introducing the methods for interpretability in depth and comprehensively as a methodological reference for future researchers or clinical practitioners in this field. Besides the methods' details, we also include a discussion of advantages and disadvantages of these methods and which scenarios each of them is suitable for, so that interested readers can know how to compare and choose among them for use. Moreover, we discuss how these methods, originally developed for solving general-domain problems, have been adapted and applied to healthcare problems and how they can help physicians better understand these data-driven technologies. Overall, we hope this survey can help researchers and practitioners in both artificial intelligence and clinical fields understand what methods we have for enhancing the interpretability of their DL models and choose the optimal one accordingly.

Theory and Practice of Integrating Machine Learning and Conventional Statistics in Medical Data Analysis

“…Deep learning, being one of the unprecedented technical advances in healthcare research, assists clinicians in understanding the role of artificial intelligence in clinical decision making. Hence, deep learning could serve as a vehicle for the translation of modern biomedical data, including electronic health records, imaging, omics, sensor data and text, which are complex, heterogeneous, poorly annotated and generally unstructured, to bridge clinical research and human interpretability [ 101 , 102 ].…”

Explainable Artificial Intelligence Methods in Combating Pandemics: A Systematic Review

“…In local interpretation methods, XAI methods attempt to accurately describe individual sample predictions as the sum of feature effects; for example, LIME explains individual predictions by replacing a locally interpretable surrogate model for the complex model; Shapley values attempt to fairly assign the prediction to individual features. In contrast to local interpretation methods, global methods such as SHAP feature importance, coefficient of regression models, and permutationbased feature importance are frequently expressed as expected values based on the distribution of the data in order to investigate the knowledge encoded in the model and its effect on predictions [117]. Depending on the scope of the problem, clinicians may consider different levels of interpretability.…”

A Coarse-to-fine Morphological Approach With Knowledge-based Rules and Self-adapting Correction for Lung Nodules Segmentation

“…Outlining juxtapleural nodules precisely is difficult since the contours may contain the lung walls, reducing the thresholding method's performance. Several people used morphological methods [17] such as the global thresholding method [18] or deep neural networks [19] to segment the lung. However, using the entire CT image as input, these methods need a lot of processing and aren't good enough to eliminate all lung wall cells.…”

“…On the other hand, deep learning models' interpretability and generalizability have been a significant barrier to their widespread adoption in healthcare settings. For instance, its performance is highly dependent on the designs and parameters of neural networks [16]; a model that performs well on one dataset may perform horribly on another one; there is no obvious rationale connecting the data about a case to the model's judgments [17]. Moreover, the interpretability is very critical for doctors to make clinical decisions.…”