HR-CAM: Precise Localization of Pathology Using Multi-level Learning in CNNs

Abstract

We propose a CNN based technique that aggregates feature maps from its multiple layers that can localize abnormalities with greater details as well as predict pathology under consideration. Existing class activation mapping (CAM) techniques extract feature maps from either the final layer or a single intermediate layer to create the discriminative maps and then interpolate to upsample to the original image resolution. In this case, the subject specific localization is coarse and is unable to capture subtle abnormalities. To mitigate this, our method builds a novel CNN based discriminative localization model that we call high resolution CAM (HR-CAM), which accounts for layers from each resolution, therefore facilitating a comprehensive map that can delineate the pathology for each subject by combining low-level, intermediate as well as highlevel features from the CNN. Moreover, our model directly provides the discriminative map in the resolution of the original image facilitating finer delineation of abnormalities. We demonstrate the working of our model on a simulated abnormalities data where we illustrate how the model captures finer details in the final discriminative maps as compared to current techniques. We then apply this technique: (1) to classify ependymomas from grade IV glioblastoma on T1weighted contrast enhanced (T1-CE) MRI and (2) to predict Parkinson's disease from neuromelanin sensitive MRI. In all these cases we demonstrate that our model not only predicts pathologies with high accuracies, but also creates clinically interpretable subject specific high resolution discriminative localizations. Overall, the technique can be generalized to any CNN and carries high relevance in a clinical setting.

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

“…Visualization-based methods provide easy-to-understand illustrations by overlaying the original images with additional visual layouts generated from transparency techniques. There existed two main visualization-based methods: (1) Visualizing pixel-attribution maps: These maps may be generated using gradient-based importance analysis 94 , 95 , pixel-level predicted probability 96 , or a combination of different levels of feature maps 97 , 98 . (2) Latent feature evolution: Encoded features were evolved according to the gradient ascent direction so that the decoded image (e.g., generated with an auto-encoder technique 99 ) gradually change from one class to another 100 , 101 .…”

“…The second approach attempted to quantify the quality of explanations for a specific purpose (functionally-grounded evaluation 123 ). For instance, some articles evaluated the localization ability of post-hoc explanations by defining an auxiliary task, such as detection 57 , 88 or segmentation 62 , 85 , 98 , 112 of anatomical structures related to the main task. They then contrasted relevant regions identified by the model with ground truth annotations.…”

“…Image visualization-based techniques to achieve transparency were most commonly considered in image classification problems. Common ways of retrieving relevance information were: Visual relevancy through attention 55 – 64 , 66 – 68 ; region occlusion by blank areas 109 , 111 or healthy-looking regions 112 ; and other techniques such as supervision of activated image regions by clinically relevant areas 88 , 89 , 92 , 94 , 95 , 97 , 98 , and image similarity 96 . Feature-based computer vision transparency priors focused on the impact of feature evolution or perturbation on the decoded output.…”

Embryo Grading With Unreliable Labels Due to Chromosome Abnormalities by Regularized PU Learning With Ranking

“…Interpretability [20], [21] of DNNs is important in clinical practice [22]- [24] as well as other real-world problems.…”

“…Interpretability (a.k.a. visual attention) of DNNs [20], [21] is a major issue especially in clinical practice (e.g., Alzheimer's disease classification [22], 3D imaging data [23], and pathology localization [24]). While these methods visualize attentions in a still image, our method improves the consistency of spatio-temporal visual attentions in time-lapse embryo images.…”