Saliency Methods for Explaining Adversarial Attacks

Gu, Jindong; Tresp, Volker

doi:10.48550/arxiv.1908.08413

Cited by 12 publications

(13 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, we propose to accomplish this by leveraging a common understanding of adversarial training. That is, the ground-truth predictions are stabilized during training and thus may have a smoother loss surface and smaller local Lipschitzness Ross and Doshi-Velez (2018); Cissé et al (2017); Chan et al (2020); Gu and Tresp (2019); Wu et al (2020a). This protection on the ground-truth class broadly exists in the vicinity of natural examples as well as the adversarial ones.…”

Section: Introductionmentioning

confidence: 99%

Attacking Adversarial Attacks as A Defense

Wu,

Pan,

Shen

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

It is well known that adversarial attacks can fool deep neural networks with imperceptible perturbations. Although adversarial training significantly improves model robustness, failure cases of defense still broadly exist. In this work, we find that the adversarial attacks can also be vulnerable to small perturbations. Namely, on adversarially-trained models, perturbing adversarial examples with a small random noise may invalidate their misled predictions. After carefully examining state-of-the-art attacks of various kinds, we find that all these attacks have this deficiency to different extents. Enlightened by this finding, we propose to counter attacks by crafting more effective defensive perturbations. Our defensive perturbations leverage the advantage that adversarial training endows the ground-truth class with smaller local Lipschitzness. By simultaneously attacking all the classes, the misled predictions with larger Lipschitzness can be flipped into correct ones. We verify our defensive perturbation with both empirical experiments and theoretical analyses on a linear model. On CIFAR10, it boosts the state-of-the-art model from 66.16% to 72.66% against the four attacks of AutoAttack, including 71.76% to 83.30% against the Square attack. On ImageNet, the top-1 robust accuracy of FastAT is improved from 33.18% to 38.54% under the 100-step PGD attack.

show abstract

Section: Introductionmentioning

confidence: 99%

Attacking Adversarial Attacks as A Defense

Wu,

Pan,

Shen

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These intentionally perturbed inputs trick the attacked models into misclassifications. Indeed, explanation methods have already proven helpful in understanding adversarial examples and dataset-related issues in the context of image analysis [7,8], and have the potential to provide similar insights in support of recent analyses on audio adversarial examples for ASR [9,10,11,12,13].…”

Section: Introductionmentioning

confidence: 80%

Visualizing Automatic Speech Recognition -- Means for a Better Understanding?

Markert,

Parracone,

Kulakov

et al. 2022

Preprint

View full text Add to dashboard Cite

Automatic speech recognition (ASR) is improving ever more at mimicking human speech processing. The functioning of ASR, however, remains to a large extent obfuscated by the complex structure of the deep neural networks (DNNs) they are based on. In this paper, we show how so-called attribution methods, that we import from image recognition and suitably adapt to handle audio data, can help to clarify the working of ASR. Taking DeepSpeech, an end-to-end model for ASR, as a case study, we show how these techniques help to visualize which features of the input are the most influential in determining the output. We focus on three visualization techniques: Layer-wise Relevance Propagation (LRP), Saliency Maps, and Shapley Additive Explanations (SHAP). We compare these methods and discuss potential further applications, such as in the detection of adversarial examples.

show abstract

“…In this section, we visualize and analyze models' attention to understand the different robustness performance of DeiT and ResNet against patch-wise perturbations. Although there are many existing methods, e.g., [14,15,34,36,44], designed for CNNs to generate saliency maps, it is not clear yet how suitable to generalize them to vision transformers. Therefore, we follow [21] to choose the model-agnostic vanilla gradient visualization method to compare the gradient (saliency) map [43] of DeiT and ResNet.…”

Section: Understanding the Robustness Of Vit To Patch-wise Perturbationsmentioning

confidence: 99%

Are Vision Transformers Robust to Patch Perturbations?

Gu¹,

Tresp²,

Qin³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The recent advances in Vision Transformer (ViT) have demonstrated its impressive performance in image classification, which makes it a promising alternative to Convolutional Neural Network (CNN). Unlike CNNs, ViT represents an input image as a sequence of image patches. The patch-wise input image representation makes the following question interesting: How does ViT perform when individual input image patches are perturbed with natural corruptions or adversarial perturbations, compared to CNNs? In this work, we study the robustness of vision transformers to patch-wise perturbations. Surprisingly, we find that vision transformers are more robust to naturally corrupted patches than CNNs, whereas they are more vulnerable to adversarial patches. Furthermore, we conduct extensive qualitative and quantitative experiments to understand the robustness to patch perturbations. We have revealed that ViT's stronger robustness to natural corrupted patches and higher vulnerability against adversarial patches are both caused by the attention mechanism. Specifically, the attention model can help improve the robustness of vision transformers by effectively ignoring natural corrupted patches. However, when vision transformers are attacked by an adversary, the attention mechanism can be easily fooled to focus more on the adversarially perturbed patches and cause a mistake.

show abstract

Saliency Methods for Explaining Adversarial Attacks

Cited by 12 publications

References 10 publications

Attacking Adversarial Attacks as A Defense

Attacking Adversarial Attacks as A Defense

Visualizing Automatic Speech Recognition -- Means for a Better Understanding?

Are Vision Transformers Robust to Patch Perturbations?

Contact Info

Product

Resources

About