Analysis of classifiers’ robustness to adversarial perturbations

Abstract

The goal of this paper is to analyze an intriguing phenomenon recently discovered in deep networks, namely their instability to adversarial perturbations (Szegedy et al, 2014). We provide a theoretical framework for analyzing the robustness of classifiers to adversarial perturbations, and show fundamental upper bounds on the robustness of classifiers. Specifically, we establish a general upper bound on the robustness of classifiers to adversarial perturbations, and then illustrate the obtained upper bound on two practical classes of classifiers, namely the linear and quadratic classifiers. In both cases, our upper bound depends on a distinguishability measure that captures the notion of difficulty of the classification task. Our results for both classes imply that in tasks involving small distinguishability, no classifier in the considered set will be robust to adversarial perturbations, even if a good accuracy is achieved. Our theoretical framework moreover suggests that the phenomenon of adversarial instability is due to the low flexibility of classifiers, compared to the difficulty of the classification task (captured mathematically by the distinguishability measure). Moreover, we further show the existence of a clear distinction between the robustness of a classifier to random noise and its robustness to adversarial perturbations. Specifically, the former is shown to be larger than the latter by a factor that is proportional to √ d (with d being the signal dimension) for linear classifiers. This result gives a theoretical explanation for the discrepancy between the two robustness properties in high dimensional problems, which was empirically observed in Szegedy et al (2014) in the context of neural networks. To the best of our knowledge, our results provide the first theoretical work that addresses the phenomenon of adversarial instability recently observed for deep networks. We finally show experimental results on controlled and real-world data that confirm the theoretical analysis and extends its spirit to more complex classification schemes.

Semantic Adversarial Attacks: Parametric Transformations That Fool Deep Classifiers

“…Moosavi-Dezfooli et al [41] prove the existence of an image-agnostic adversarial perturbation. Fawzi et al [15] extend this to theoretically show that every classifier is vulnerable to adversarial attacks. Moosavi-Dezfooli et alfurther consider the effect of the curvature of the decision boundaries on the existence of adversarial examples in [43].…”

Universal Adversarial Perturbations

“…The robustness of image classifiers to structured and unstructured perturbations have recently attracted a lot of attention [19,16,20,3,4,12,13,14]. Despite the impressive performance of deep neural network architectures on challenging visual classification benchmarks [6,9,21,10], these classifiers were shown to be highly vulnerable to perturbations.…”

Introducing Graph Smoothness Loss for Training Deep Learning Architectures

“…However, this drop is compensated by a better accommodation to deviations of the inputs, as reported by the Mean Corruption Error (MCE) scores (see [28]). Such a trade-off between accuracy and robustness has been discussed in [29]. For this experiment, we fixed k to its maximum value, d = 200, α = 2 and we used 10-NN as a classifier when using the graph smoothness loss.…”