Explainable AI for CNN-based prostate tumor segmentation in multi-parametric MRI correlated to whole mount histopathology

Abstract

Automatic prostate tumor segmentation is often unable to identify the lesion even if multi-parametric MRI data is used as input, and the segmentation output is difficult to verify due to the lack of clinically established ground truth images. In this work we use an explainable deep learning model to interpret the predictions of a convolutional neural network (CNN) for prostate tumor segmentation. The CNN uses a U-Net architecture which was trained on multi-parametric MRI data from 122 patients to automatically segment the prostate gland and prostate tumor lesions. In addition, co-registered ground truth data from whole mount histopathology images were available in 15 patients that were used as a test set during CNN testing. To be able to interpret the segmentation results of the CNN, heat maps were generated using the Gradient Weighted Class Activation Map (Grad-CAM) method. The CNN achieved a mean Dice Sorensen Coefficient 0.62 and 0.31 for the prostate gland and the tumor lesions -with the radiologist drawn ground truth and 0.32 with whole-mount histology ground truth for tumor lesions. Dice Sorensen Coefficient between CNN predictions and manual segmentations from MRI and histology data were not significantly different. In the prostate the Grad-CAM heat maps could differentiate between tumor and healthy prostate tissue, which indicates that the image information in the tumor was essential for the CNN segmentation.

COVLIAS 2.0-cXAI: Cloud-Based Explainable Deep Learning System for COVID-19 Lesion Localization in Computed Tomography Scans

“…That poses an intriguing question: how can you trust a model’s decisions if you cannot fully justify how it got there? There has been the latest trend in the growth of XAI for a better understanding of the AI black boxes [ 49 , 136 , 137 , 138 , 139 ]. Grad-CAM or Grad-CAM++ produces a coarse clustering map showing the key regions in the picture for predicting any target idea (say, “COVID-19” in a classification network) by using the gradients of any target concept (say, “COVID-19” in a classification network) in the final convolutional layer.…”

“…New techniques have evolved such as SHAP [ 52 , 158 ] and UMAP [ 159 ]. Heatmaps produced by Grad-CAM have been used for XAI in several applications [ 64 ], where the generated heatmaps are the threshold to compute the lesions which are then compared against the gold standard [ 49 ]. Choi et al [ 48 ] used SHAP to demonstrate the high-risk factors responsible for higher phosphate.…”

“…Thus, there is a clear need for human readability and interpretability of deep networks, which requires identified lesions to be interpreted and quantified. We, therefore, developed an explainable AI system in a cloud framework, labeled the “COVLIAS 2.0-cXAI” system, which was our primary novelty [ 47 , 48 , 49 , 50 , 51 , 52 ]. The COVLIAS 2.0-cXAI design consisted of three stages ( Figure 1 ): (i) automated lung segmentation using the hybrid deep learning ResNet-UNet model using automatic adjustment of Hounsfield units [ 53 ], hyperparameter optimization [ 54 ], and the parallel and distributed nature of design during training; (ii) classification using three kinds of DenseNet (DN) models (DN-121, DN-169, DN-201) [ 55 , 56 , 57 , 58 ]; and (iii) scientific validation using four kinds of class activation mapping (CAM) visualization techniques: gradient-weighted class activation mapping (Grad-CAM) [ 59 , 60 , 61 , 62 , 63 ], Grad-CAM++ [ 64 , 65 , 66 , 67 ], score-weighted CAM (Score-CAM) [ 68 , 69 , 70 ], and FasterScore-CAM [ 71 , 72 ].…”

Multi Level Approach for Segmentation of Interstitial Lung Disease (ILD) Patterns Classification Based on Superpixel Processing and Fusion of K-Means Clusters: SPFKMC

“…In the literature, there are many types of segmentation algorithms applied to medical images, such as thresholding [ 10 , 11 ], region growing [ 12 , 13 ], machine learning [ 14 , 15 ], deep-learning [ 16 , 17 ], active contour [ 18 , 19 ], quantum-inspired computing [ 20 , 21 ], and computational intelligence [ 22 , 23 ]. Therefore, this section sequentially and individually reviews the related recent developments in unsupervised and supervised categories.…”

“…Researcher in [ 2 ] has introduced a deep learning algorithm for COVID-19-inspired lung image segmentation extended to [ 16 , 17 ]. They employed SegNet and the U -Net network to demonstrate deep learning-based infected tissue segmentation in lung images.…”