Interpretable Image Recognition by Constructing Transparent Embedding Space

Wang, Jiaqi; Liu, Huafeng; Wang, Xinyue; Jing, Liping

doi:10.1109/iccv48922.2021.00093

Cited by 63 publications

(61 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ProtoPNet uses the similarity scores between an input image and the learned prototypes to generate explanations for its predictions in the form of "this looks like that," as in Figure 1(a). The ProtoPNet model has been extended multiple times [28,29,46]. We build our Deformable Pro-toPNet upon the ProtoPNet and TesNet models [46].…”

Section: Related Workmentioning

confidence: 99%

“…The ProtoPNet model has been extended multiple times [28,29,46]. We build our Deformable Pro-toPNet upon the ProtoPNet and TesNet models [46]. Tes-Net uses a cosine similarity metric to compute similarities between image patches and prototypes in a latent space, and introduces loss terms to encourage the prototype vectors within a class to be orthogonal to each other and to separate the latent spaces of different classes.…”

Section: Related Workmentioning

confidence: 99%

“…By doing so we aim to learn a useful feature space where the image features ẑ∆ a,b of inputs of class c are clustered around prototypes p(c,l) of the same class, but separated from those of other classes on a hypersphere. To achieve this, we use the cluster and separation losses as in [5] and adapted for the angular space in [46]. The cluster and separation losses are defined as:…”

Section: Trainingmentioning

confidence: 99%

“…In computer vision, there is a growing stream of research that aims to address this problem by integrating the power of deep neural networks and the interpretability of case-based reasoning to produce accurate yet interpretable image classification models [3,5,29,46]. These models learn a set of prototypes from training images.…”

Section: Introductionmentioning

confidence: 99%

“…(2) We improved the accuracy of case-based interpretable neural networks by introducing angular margins to the training algorithm. (3) We showed that, unlike its predecessors that use spatially rigid prototypes [5,46], a Deformable ProtoPNet can be easily trained using full (not cropped) images from the CUB-200-2011 bird recognition dataset [48], and can achieve competitive accuracy on full test images.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes

Donnelly¹,

Barnett²,

Chen³

2021

Preprint

View full text Add to dashboard Cite

Machine learning has been widely adopted in many domains, including high-stakes applications such as healthcare, finance, and criminal justice. To address concerns of fairness, accountability and transparency, predictions made by machine learning models in these critical domains must be interpretable. One line of work approaches this challenge by integrating the power of deep neural networks and the interpretability of case-based reasoning to produce accurate yet interpretable image classification models. These models generally classify input images by comparing them with prototypes learned during training, yielding explanations in the form of "this looks like that." However, methods from this line of work use spatially rigid prototypes, which cannot explicitly account for pose variations. In this paper, we address this shortcoming by proposing a casebased interpretable neural network that provides spatially flexible prototypes, called a deformable prototypical part network (Deformable ProtoPNet). In a Deformable Pro-toPNet, each prototype is made up of several prototypical parts that adaptively change their relative spatial positions depending on the input image. This enables each prototype to detect object features with a higher tolerance to spatial transformations, as the parts within a prototype are allowed to move. Consequently, a Deformable ProtoPNet can explicitly capture pose variations, improving both model accuracy and the richness of explanations provided. Compared to other case-based interpretable models using prototypes, our approach achieves competitive accuracy, gives an explanation with greater context, and is easier to train, thus enabling wider use of interpretable models for computer vision.

show abstract