Attack to Fool and Explain Deep Networks

Abstract

Deep visual models are susceptible to adversarial perturbations to inputs. Although these signals are carefully crafted, they still appear noise-like patterns to humans. This observation has led to the argument that deep visual representation is misaligned with human perception. We counter-argue by providing evidence of human-meaningful patterns in adversarial perturbations. We first propose an attack that fools a network to confuse a whole category of objects (source class) with a target label. Our attack also limits the unintended fooling by samples from non-sources classes, thereby circumscribing human-defined semantic notions for network fooling. We show that the proposed attack not only leads to the emergence of regular geometric patterns in the perturbations, but also reveals insightful information about the decision boundaries of deep models. Exploring this phenomenon further, we alter the 'adversarial' objective of our attack to use it as a tool to 'explain' deep visual representation. We show that by careful channeling and projection of the perturbations computed by our method, we can visualize a model's understanding of human-defined semantic notions. Finally, we exploit the explanability properties of our perturbations to perform image generation, inpainting and interactive image manipulation by attacking adversarialy robust 'classifiers'. In all, our major contribution is a novel pragmatic adversarial attack that is subsequently transformed into a tool to interpret the visual models. The article also makes secondary contributions in terms of establishing the utility of our attack beyond the adversarial objective with multiple interesting applications.

Benchmarking Adversarial Patch Against Aerial Detection

“…Finally, we summarize physical attacks against image classification ( [17], [86], [90]- [93], [156], [162], [164], [165], [167]- [171]) in Table IV.…”

Black-box and Target-specific Attack Against Interpretable Deep Learning Systems

“…However, the universal perturbation can still be a noise-like pattern to the human eye when the interpretation is applied. The study [10] proposed that extended universal perturbation can exploit the explainability of models by carefully exploring the decision boundaries of deep models. The authors showed that their attack can be used to interpret the internal working process of DL models.…”

“…When an interpreter is adopted with a DL model, the involvement of the adversary can easily be detected (see Figure 1). The main idea of our attack is also based on generating universal perturbation [10]. Unlike those attacks, we consider misleading the interpretability along with classification while generating adversarial samples to increase the robustness and stealthiness of the attack.…”

“…Thus, Interpretable Deep Learning Systems (IDLSes) have gained considerable attention, as they provide predictions and interpretations. However, recent studies have demonstrated the feasibility and practicality of creating adversarial examples (AE) that deceive both the target prediction model and its associated interpreters [8], [9], [10], [11], [12]. Consequently, IDLSes cannot guarantee robust security measures for detecting AEs.…”

“…Additionally, Figure 2 demonstrates that the employed interpreter can detect previous attacks, even in white-box scenarios. For example, Figure 1 shows that the existing universal perturbation attacks [10] can be detected due to the inconsistencies between the adversarial interpretations and the object of the image, which can be recognized by an "observer." On the other hand, our approach generates adversarial interpretations that are indistinguishable from benign ones, suggesting no manipulation of the input data.…”