Sparse Activations for Interpretable Disease Grading

Abstract

Interpreting deep learning models typically relies on post-hoc saliency map techniques. However, these techniques often fail to serve as actionable feedback to clinicians, and they do not directly explain the decision mechanism. Here, we propose an inherently interpretable model that combines the feature extraction capabilities of deep neural networks with advantages of sparse linear models in interpretability. Our approach relies on straightforward but effective changes to a deep bag-of-local-features model (BagNet). These modifications lead to fine-grained and sparse class evidence maps which, by design, correctly reflect the model's decision mechanism. Our model is particularly suited for tasks which rely on characterising regions of interests that are very small and distributed over the image. In this paper, we focus on the detection of Diabetic Retinopathy, which is characterised by the progressive presence of small retinal lesions on fundus images. We observed good classification accuracy despite our added sparseness constraint. In addition, our model precisely highlighted retinal lesions relevant for the disease grading task and excluded irrelevant regions from the decision mechanism. The results suggest our sparse BagNet model can be a useful tool for clinicians as it allows efficient inspection of the model predictions and facilitates clinicians' and patients' trust.

An Inherently Interpretable AI model improves Screening Speed and Accuracy for Early Diabetic Retinopathy

“…One limitation of our model is that it may not provide good explanations if its inductive bias is not matched to the disease, e.g. when lesions cover large parts of the retina as in more advanced DR grades [17].…”

“…Architecture. We trained and evaluated an inherently interpretable deep convolutional neural network (sparse BagNet [17,18]) for early DR detection. The sparse BagNet is an implicitly patch-based model based on bag-of-local features and aggregates local evidence from interpretable heatmaps to make predictions (figure 1b).…”

“…Criteria for evaluating the performance of the inherently interpretable deep learning model were specified before the start of the study based on previous work [17]. We evaluated three aspects of the model's quality: DR screening performance.…”

“…We address this issue and validate an inherently interpretable approach for screening for early DR in a retrospective reader study. Our approach uses a deep learning architecture called sparse BagNets [17,18], which explicitly models the local evidence for the presence of DR as part of its network architecture (figure 1b). Most studies so far have considered the task of screening for moderate non-proliferative DR or more advanced stages [7], although even mild non-proliferative diabetic retinopathy (NPDR) is recommended for close monitoring and careful control of hyperglycemia [5,19].…”