NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weights

Abstract

Attribution methods remain a practical instrument that is used in real-world applications to explain the decision-making process of complex learning machines. It has been shown that a simple method called SmoothGrad can effectively reduce the visual diffusion of gradient-based attribution methods and has established itself among both researchers and practitioners. What remains unexplored in research, however, is how explanations can be improved by introducing stochasticity to the model weights. In the light of this, we introduce -NoiseGrad -a stochastic, method-agnostic explanation-enhancing method that adds noise to the weights instead of the input data. We investigate our proposed method through various experiments including different datasets, explanation methods and network architectures and conclude that NoiseGrad (and its extension NoiseGrad++) with multiplicative Gaussian noise offers a clear advantage compared to SmoothGrad on several evaluation criteria. We connect our proposed method to Bayesian Learning and provide the user with a heuristic for choosing hyperparameters.

Explaining Bayesian Neural Networks

“…Most of the work on the topic of interpreting the BNNs concentrate on uncertainty quantification and visualization: [66] proposes a method to decompose the momentbased predictive uncertainty into two parts: aleatoric and epistemic: in [67] author proposes a model-agnostic method to visualize the contribution of individual features to predictive, epistemic and aleatoric uncertainty [66]. Recently, it has been shown that explanations of DNN can be enhanced by introducing stochasticities to the model weights [68], which, to some extend, lead to explanations similar to the ones using Diagonal or KFAC Laplace approximation. The so-called NoiseGrad method [68] adds multiplicative Gaussian noise to the model weights, which significantly reduces the gradient shattering effect [69] similar to the SmoothGrad method [70].…”

“…Recently, it has been shown that explanations of DNN can be enhanced by introducing stochasticities to the model weights [68], which, to some extend, lead to explanations similar to the ones using Diagonal or KFAC Laplace approximation. The so-called NoiseGrad method [68] adds multiplicative Gaussian noise to the model weights, which significantly reduces the gradient shattering effect [69] similar to the SmoothGrad method [70].…”

“…Hence, exemplary, if the prediction of a model is "cat", we assume that in the given image parts of the object cat are responsible for the prediction and subsequently yield positive relevance attribution by the explanation method. Thus, in order to correctly measure the ability of the method to "find" the object of interest, ground-truth segmentations are required [74], [75], [76], [77], [78], [79], [68].…”

“…We evaluate our proposed methods on the Custom MNIST (CMNIST) dataset [68], where MNIST digits are plotted on a randomly chosen CIFAR background. This ensures that the decision-making basis for the model should rely only on the number and not on the randomly chosen background.…”

“…Fig.4: Illustrative explanations of a Bayesian Neural Network. The BNN was trained with Dropout on the Custom MNIST dataset[68]. The input, an MNIST digit zero on a random CIFAR Background is shown on the left and was correctly classified as zero by the BNN.…”