Resolving challenges in deep learning-based analyses of histopathological images using explanation methods

Abstract

Deep learning has recently gained popularity in digital pathology due to its high prediction quality. However, the medical domain requires explanation and insight for a better understanding beyond standard quantitative performance evaluation. Recently, explanation methods have emerged, which are so far still rarely used in medicine. This work shows their application to generate heatmaps that allow to resolve common challenges encountered in deep learning-based digital histopathology analyses. These challenges comprise biases typically inherent to histopathology data. We study binary classification tasks of tumor tissue discrimination in publicly available haematoxylin and eosin slides of various tumor entities and investigate three types of biases:(1) biases which affect the entire dataset, (2) biases which are by chance correlated with class labels and (3) sampling biases. While standard analyses focus on patch-level evaluation, we advocate pixel-wise heatmaps, which offer a more precise and versatile diagnostic instrument and furthermore help to reveal biases in the data. This insight is shown to not only detect but also to be helpful to remove the effects of common hidden biases, which improves generalization within and across datasets. For example, we could see a trend of improved area under the receiver operating characteristic curve by 5% when reducing a labeling bias. Explanation techniques are thus demonstrated to be a helpful and highly relevant tool for the development and the deployment phases within the life cycle of real-world applications in digital pathology. Related work Models in digital pathologySimilar to the recent trend in computer vision, where end-to-end training with deep learning clearly prevails classification of handcrafted features, an increased use of deep learning, e.g convolutional neural networks (CNN) is also noticeable in digital pathology. Nonetheless, there are some works on combining support vector machines with image feature algorithms 21, 23-25 . Meanwhile, while some works propose custom-designed networks 26, 27 , e. g. spatially constrained, locality sensitive CNN for the detection of nuclei in histopathological images 26 , most often standard deep learning architectures (e. g. AlexNet 28 , GoogLeNet 3 , ResNet 29 ) as well as hybrids are used for digital pathology 22, 30-33 . According to 6 , currently the most common architecture is the GoogLeNet Inception-V3 model. Interpretability in computational pathologyAs discussed above, more and more developments have emerged that introduce the possibility of explanation (e. g. [11][12][13][14][15][16][17][18] ; for a summary of implementations see 34 ); few of which have been applied in digital pathology 21,22,30,[35][36][37][38][39][40] . The visualization of a support vector machine's decision on Bag-of-Visual-Words features in a histopathological discrimination task is explored in 21 . The authors present an explanatory approach for evidence of tumor and lymphocytes in H&E images as well as for molecular properties which-unli...

Unraveling the deep learning gearbox in optical coherence tomography image segmentation towards explainable artificial intelligence

“…In fact, it can be applied to improve the understanding of any machine learning algorithm that learns from ambiguous ground truth data. For example, T-REX could be used in the application of uncovering biases of ML prediction models in digital histopathology not only with respect to data set biases but also with respect to varying opinions of experts labeling the histopathology images 80 . In applications, where supervised ML decision models are trained to detect diseases such as Covid-19 (ref.…”

The Challenges of Machine Learning and Their Economic Implications

“…This, in turn, could affect the demand: if sufficient trust is not provided, customers may opt not to use these models. In general, this attitude of the demand side of the market incentivizes organizations to optimally fund research to develop interpretation techniques with the aim of identifying and removing biases and generating trust [62]. Consequently, the incentives of all the involved stakeholders are expected to be aligned and lead to the maximization of aggregate social welfare.…”

The Next Generation of Medical Decision Support: A Roadmap Toward Transparent Expert Companions

“…In addition to purely focusing on prediction accuracy, the authors applied LRP in order to analyze the nonlinear properties of the learning machine by mapping the results of a prediction onto a heatmap that reveals the morphological particularities of the studied pathological properties. Hägele et al (2019) analyzed histopathological images and applied LRP for visual and quantitative verification of features used for prediction as well as for detection of various latent but crucial biases using heatmapping. Out of such explanations and visualizations, experts might get valuable interpretations, but to even improve interpretability especially for lay humans it could be helpful to include other explanation modalities.…”