IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammography

Abstract

Interpretability in machine learning models is important in high-stakes decisions, such as whether to order a biopsy based on a mammographic exam. Mammography poses important challenges that are not present in other computer vision tasks: datasets are small, confounding information is present, and it can be difficult even for a radiologist to decide between watchful waiting and biopsy based on a mammogram alone. In this work, we present a framework for interpretable machine learning-based mammography. In addition to predicting whether a lesion is malignant or benign, our work aims to follow the reasoning processes of radiologists in detecting clinically relevant semantic features of each image, such as the characteristics of the mass margins. The framework includes a novel interpretable neural network algorithm that uses casebased reasoning for mammography. Our algorithm can incorporate a combination of data with whole image labelling and data with pixel-wise annotations, leading to better accuracy and interpretability even with a small number of images. Our interpretable models are able to highlight the classification-relevant parts of the image, whereas other methods highlight healthy tissue and confounding information. Our models are decision aids, rather than decision makers, aimed at better overall human-machine collaboration. We do not observe a loss in mass margin classification accuracy over a black box neural network trained on the same data.

Interpretable Image Classification with Differentiable Prototypes Assignment

“…Approaches similar to ProtoPNet organize the prototypes hierarchically [19] to classify input at every level of a predefined taxonomy or transform prototypes from the latent space to data space [31]. Lastly, prototype-based solutions are widely adopted in various fields like medical imaging [3,5,27,52], time series analysis [15], and sequence learning [36].…”

“…We use two datasets: CUB-200-2011 [56] consisted of 200 species of birds and Stanford Cars [30] with 196 car models. For both datasets, images are augmented offline using parameters from Table 5, and the process of data preparation is the same as in [8] 5 .…”

Toward a Unified Framework for Debugging Concept-based Models

“…The concepts and the weights are learned sequentially. [18] specializes PPNets to image-based medical diagnosis. In particular, IAIA-BL accepts per-example attribute relevance information (e.g., annotations of symptomatic regions in X-ray images) and penalizes part prototypes that activate outside the relevant areas.…”

“…To this end, [27] propose a perturbation-based technique -analogous to LIME [28] -for explaining why a particular prototype activates on a certain region of an image, but it does not illustrate how to fix this kind of bugs. Finally, [18] introduce a loss term for PPNets that penalizes concepts that activate on regions annotated as irrelevant by a domain expert. None of these works, however, considers interaction with a human debugger and the issues that this brings with it.…”

“…GBMs explain their own predictions by supplying faithful concept-level attribution maps -which encompass both the concepts and the aggregation weights -, thus facilitating the identification of bugs affecting the model. Work on troubleshooting of GBMs is, however, sparse and ad hoc [16]: some approaches assume the concepts to be given and focus on correcting the aggregation step [4,17], while others fix the learned concepts while ignoring how they are aggregated [18,15]. Fixing only one set of bugs is however insufficient.…”