The (Un)reliability of Saliency Methods

Kindermans, Pieter-Jan; Hooker, Sara; Adebayo, Julius; Alber, Maximilian; Schütt, Kristof T.; Dähne, Sven; Erhan, Dumitru; Kim, Been

doi:10.1007/978-3-030-28954-6_14

Cited by 420 publications

(358 citation statements)

References 9 publications

Supporting

Mentioning

354

Contrasting

Unclassified

Order By: Relevance

“…Understanding and visualizing networks. Most prior work on network visualization concerns discriminative classifiers [1,3,23,26,37,38,48,50]. GANs have been visualized by examining the discriminator [32] and the semantics of internal features [4].…”

Section: Related Workmentioning

confidence: 99%

Seeing What a GAN Cannot Generate

Bau

Zhu²,

Wulff³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

329

205

View full text Add to dashboard Cite

Despite the success of Generative Adversarial Networks (GANs), mode collapse remains a serious issue during GAN training. To date, little work has focused on understanding and quantifying which modes have been dropped by a model. In this work, we visualize mode collapse at both the distribution level and the instance level. First, we deploy a semantic segmentation network to compare the distribution of segmented objects in the generated images with the target distribution in the training set. Differences in statistics reveal object classes that are omitted by a GAN. Second, given the identified omitted object classes, we visualize the GAN's omissions directly. In particular, we compare specific differences between individual photos and their approximate inversions by a GAN. To this end, we relax the problem of inversion and solve the tractable problem of inverting a GAN layer instead of the entire generator. Finally, we use this framework to analyze several recent GANs trained on multiple datasets and identify their typical failure cases.

show abstract

Section: Related Workmentioning

confidence: 99%

Seeing What a GAN Cannot Generate

Bau

Zhu²,

Wulff³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

329

205

View full text Add to dashboard Cite

show abstract

“…A rich set of image-based saliency methods have been developed over the years (e.g., [24,29,14,26,3,6]). One common approach of determining salient inputs is to rely on the changes in the model output, such as gradients of the output with respect to the input features.…”

Section: Introductionmentioning

confidence: 99%

XRAI: Better Attributions Through Regions

Kapishnikov

Bolukbasi

Viégas

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

185

168

View full text Add to dashboard Cite

Saliency methods can aid understanding of deep neural networks. Recent years have witnessed many improvements to saliency methods, as well as new ways for evaluating them. In this paper, we 1) present a novel region-based attribution method, XRAI, that builds upon integrated gradients [26], 2) introduce evaluation methods for empirically assessing the quality of image-based saliency maps (Performance Information Curves (PICs)), and 3) contribute an axiom-based sanity check for attribution methods. Through empirical experiments and example results, we show that XRAI produces better results than other saliency methods for common models and the ImageNet dataset.

show abstract

“…These attribution methods are able to integrate distributed representations throughout the network to arrive at the importance of each nucleotide variant for a given sequence. However, they only consider one sequence at a time and their scores are noisy (Kindermans et al , 2017;Adebayo et al , 2018), which can result in signicant scores for nucleotide variants that are not necessarily biologically relevant. To address these issues, TF-MoDISco splits the attribution scores of each sequence into subsequences called sequlets, clusters and aligns these sequlets, then averages each cluster to provide more interpretable representations learned by the network (Shrikumar et al , 2018).…”

Section: Introductionmentioning

confidence: 99%

Representation Learning of Genomic Sequence Motifs with Convolutional Neural Networks

Koo

Eddy

2018

Preprint

View full text Add to dashboard Cite

Although convolutional neural networks (CNNs) have been applied to a variety of computational genomics problems, there remains a large gap in our understanding of how they work. Here we perform systematic experiments on synthetic sequences to reveal principles of how CNN architecture in uences the internal representations of genomic sequence motifs that are learned. We focus our study on representations learned by rst convolutional layer lters. We nd that deep CNNs tend to learn distributed representations of partial sequence motifs. However, we demonstrate that the architecture of a CNN can be modi ed to predictively learn more interpretable localist representations, i.e. whole motifs. We then validate that the representation learning principles established from synthetic sequences generalize to in vivo sequences.

show abstract

The (Un)reliability of Saliency Methods

Cited by 420 publications

References 9 publications

Seeing What a GAN Cannot Generate

Seeing What a GAN Cannot Generate

XRAI: Better Attributions Through Regions

Representation Learning of Genomic Sequence Motifs with Convolutional Neural Networks

Contact Info

Product

Resources

About