Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps

Kim, Beomsu; Seo, Junghoon; Jeon, Seunghyeon; Koo, Jamyoung; Choe, Jeongyeol; Jeon, Taegyun

doi:10.1109/iccvw.2019.00510

Cited by 47 publications

(19 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Guided Backpropagation [43], negative gradients are set to 0, effectively discarding suppression of neuron activation. Rectified Gradient [103] generalizes this by allowing layerwise thresholding with an extra hyper parameter. Grad-CAM [104] calculates the gradient of the class score with respect to channels, i.e.…”

Section: Importance Estimators and Relevance Scoresmentioning

confidence: 99%

Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization

et al. 2021

View full text Add to dashboard Cite

Over the last decade there has been an extensive evolution in the Artificial Intelligence (AI) field. Modern radiation oncology is based on the exploitation of advanced computational methods aiming to personalization and high diagnostic and therapeutic precision. The quantity of the available imaging data and the increased developments of Machine Learning (ML), particularly Deep Learning (DL), triggered the research on uncovering "hidden" biomarkers and quantitative features from anatomical and functional medical images. Deep Neural Networks (DNN) have achieved outstanding performance and broad implementation in image processing tasks. Lately, DNNs have been considered for radiomics and their potentials for explainable AI (XAI) may help classification and prediction in clinical practice. However, most of them are using limited datasets and lack generalized applicability. In this study we review the basics of radiomics feature extraction, DNNs in image analysis, and major interpretability methods that help enable explainable AI. Furthermore, we discuss the crucial requirement of multicenter recruitment of large datasets, increasing the biomarkers variability, so as to establish the potential clinical value of radiomics and the development of robust explainable AI models.

show abstract

Section: Importance Estimators and Relevance Scoresmentioning

confidence: 99%

Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Recently, deep convolutional neural nets have been proposed to predict salience maps as their output [ 54 ]. Alternatively, “salience maps” of the deepest layers in neural networks are explored not for attention modeling, but mainly for visualization and explanatory purposes [ 55 , 56 ]. We tested two of such deep learning models: the multiduration model [ 26 ], which predicts how the duration of each observation affects salience, and the temporally-aggregating spatial encoder-decoder network (TASED) [ 32 ], which was proposed as a video-specific salience model.…”

Section: Experiments With Computational Salience Modelsmentioning

confidence: 99%

Glimpse: A Gaze-Based Measure of Temporal Salience

Traver

Zorío

Leiva

2021

Sensors

View full text Add to dashboard Cite

Temporal salience considers how visual attention varies over time. Although visual salience has been widely studied from a spatial perspective, its temporal dimension has been mostly ignored, despite arguably being of utmost importance to understand the temporal evolution of attention on dynamic contents. To address this gap, we proposed Glimpse, a novel measure to compute temporal salience based on the observer-spatio-temporal consistency of raw gaze data. The measure is conceptually simple, training free, and provides a semantically meaningful quantification of visual attention over time. As an extension, we explored scoring algorithms to estimate temporal salience from spatial salience maps predicted with existing computational models. However, these approaches generally fall short when compared with our proposed gaze-based measure. Glimpse could serve as the basis for several downstream tasks such as segmentation or summarization of videos. Glimpse’s software and data are publicly available.

show abstract

“…Despite early successes, direct calculation of the gradients often leads to noisy saliency maps without clearly focused regions. Several propositions have been made to improve them, such as Guided Backropagation (GuidedBP) [14], Rectified Gradient (RectGrad) [23] and SmoothGrad [15]. The first two methods modify the back-propagation of the gradient through the Rectified Linear Activation Unit (ReLU) f (x) = max(x, 0), for a detailed description see sec.…”

Section: B Saliency Mapsmentioning

confidence: 99%

“…and this additional guiding of the gradient leads to sharper activation maps. Recently, Rectified Gradient [23] has been proposed as yet another method to calculate gradients. It introduces an external parameter τ which acts as a threshold for gradients to be backpropagated:…”

Section: B Calculating the Gradientsmentioning

confidence: 99%

“…Concept vectors and concept scores are built from known attributes. For the example images with the highest concept scores, Guided Backpropagation[14] (second row) and Rectified Gradient[23] (third row) are used to obtain concept saliency maps. For the last column, artificial blue squares have been inserted into a subset of 10000 pictures and the VAE has been re-trained with this subset as part of the training data.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models

Brocki

Chung

2019

2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA)

View full text Add to dashboard Cite

Evaluating, explaining, and visualizing high-level concepts in generative models, such as variational autoencoders (VAEs), is challenging in part due to a lack of known prediction classes that are required to generate saliency maps in supervised learning. While saliency maps may help identify relevant features (e.g., pixels) in the input for classification tasks of deep neural networks, similar frameworks are understudied in unsupervised learning. Therefore, we introduce a new method of obtaining saliency maps for latent representations of known or novel high-level concepts, often called concept vectors in generative models. Concept scores, analogous to class scores in classification tasks, are defined as dot products between concept vectors and encoded input data, which can be readily used to compute the gradients. The resulting concept saliency maps are shown to highlight input features deemed important for high-level concepts. Our method is applied to the VAE's latent space of CelebA dataset in which known attributes such as "smiles" and "hats" are used to elucidate relevant facial features. Furthermore, our application to spatial transcriptomic (ST) data of a mouse olfactory bulb demonstrates the potential of latent representations of morphological layers and molecular features in advancing our understanding of complex biological systems. By extending the popular method of saliency maps to generative models, the proposed concept saliency maps help improve interpretability of latent variable models in deep learning.Codes to reproduce and to implement concept saliency maps: https://github.com/lenbrocki/concept-saliency-maps

show abstract

Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps

Cited by 47 publications

References 8 publications

Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization

Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization

Glimpse: A Gaze-Based Measure of Temporal Salience

Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models

Contact Info

Product

Resources

About