Enhanced Regularizers for Attributional Robustness

Sarkar, Anindya; Sarkar, Anirban; Balasubramanian, Vineeth N

doi:10.1609/aaai.v35i3.16355

Cited by 6 publications

(11 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This makes the optimization unstable. To address this issue Sarkar et al [97] used a triplet loss with softplus non-linearities. This approach focuses on pixels that attribute highly to the prediction of the true class (positive class pixels).…”

Section: Combination Of Tunedmentioning

confidence: 99%

“…So, contrastive learning cannot improve until focus is also given to negative and positive class pixels. Sarkar et al [97] applied this concept by using a regularizer that forces the true class distribution to be skew-shaped and the negative class to behave uniformly. They also tried to impose a bound to change in the pixel attribution weight as indistinguishable changes are made to the input via another regularizer.…”

Section: Combination Of Tunedmentioning

confidence: 99%

“…We first look at various aspects satisfied by an explanation map obtained from an adversarially robust model and understand the multiple connections between the properties of explanations and those satisfied by an adversarially robust model. While most studies are based on image classification, [73,97] show that their approach also addresses the robustness of attributions for other tasks like segmentation and object detection as illustrated in Figure 9.…”

Section: Connecting Explanations To Adversarial Robustnessmentioning

confidence: 99%

“…Many recent efforts have demonstrated the vulnerability of explanations (or attributions 1 ) to human-imperceptible input perturbations across image, text and tabular data [36,45,55,62,107,108,133]. Similarly, there have also been many efforts in recent years in securing the stability of such explanations in [13,26,30,36,37,50,54,73,97,99,106,125].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On the Robustness of Explanations of Deep Neural Network Models: A Survey

Jyoti¹,

Ganesh²,

Gayala³

et al. 2022

Preprint

View full text Add to dashboard Cite

Explainability has been widely stated as a cornerstone of the responsible and trustworthy use of machine learning models. With the ubiquitous use of Deep Neural Network (DNN) models expanding to risk-sensitive and safety-critical domains, many methods have been proposed to explain the decisions of these models. Recent years have also seen concerted efforts that have shown how such explanations can be distorted (attacked) by minor input perturbations. While there have been many surveys that review explainability methods themselves, there has been no effort hitherto to assimilate the different methods and metrics proposed to study the robustness of explanations of DNN models. In this work, we present a comprehensive survey of methods that study, understand, attack, and defend explanations of DNN models. We also present a detailed review of different metrics used to evaluate explanation methods, as well as describe attributional attack and defense methods. We conclude with lessons and take-aways for the community towards ensuring robust explanations of DNN model predictions.

show abstract

Section: Combination Of Tunedmentioning

confidence: 99%

Section: Combination Of Tunedmentioning

confidence: 99%

Section: Connecting Explanations To Adversarial Robustnessmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the Robustness of Explanations of Deep Neural Network Models: A Survey

Jyoti¹,

Ganesh²,

Gayala³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…(Lipton 2018;Samek et al 2019;Fan et al 2021;Zhang et al 2020) present detailed surveys on these methods. Recently, the growing numbers of attribution methods has led to a concerted focus on studying the robustness of attributions to input perturbations to handle potential security hazards (Chen et al 2019;Sarkar, Sarkar, and Balasubramanian 2021;Wang and Kong 2022;Agarwal et al 2022). One could view these efforts as akin to adversarial robustness that focuses on defending against attacks on model predictions, whereas attributional robustness focuses on defending against attacks on model explanations.…”

Section: Introductionmentioning

confidence: 99%

Rethinking Robustness of Model Attributions

Kamath,

Mittal,

Deshpande

et al. 2024

AAAI

View full text Add to dashboard Cite

For machine learning models to be reliable and trustworthy, their decisions must be interpretable. As these models find increasing use in safety-critical applications, it is important that not just the model predictions but also their explanations (as feature attributions) be robust to small human-imperceptible input perturbations. Recent works have shown that many attribution methods are fragile and have proposed improvements in either these methods or the model training. We observe two main causes for fragile attributions: first, the existing metrics of robustness (e.g., top-k intersection) overpenalize even reasonable local shifts in attribution, thereby making random perturbations to appear as a strong attack, and second, the attribution can be concentrated in a small region even when there are multiple important parts in an image. To rectify this, we propose simple ways to strengthen existing metrics and attribution methods that incorporate locality of pixels in robustness metrics and diversity of pixel locations in attributions. Towards the role of model training in attributional robustness, we empirically observe that adversarially trained models have more robust attributions on smaller datasets, however, this advantage disappears in larger datasets. Code is made available at https://github.com/ksandeshk/LENS.

show abstract