Excessive Invariance Causes Adversarial Vulnerability

Abstract

Despite their impressive performance, deep neural networks exhibit striking failures on out-of-distribution inputs. One core idea of adversarial example research is to reveal neural network errors under such distribution shifts. We decompose these errors into two complementary sources: sensitivity and invariance. We show deep networks are not only too sensitive to task-irrelevant changes of their input, as is well-known from -adversarial examples, but are also too invariant to a wide range of task-relevant changes, thus making vast regions in input space vulnerable to adversarial attacks. We show such excessive invariance occurs across various tasks and architecture types. On MNIST and ImageNet one can manipulate the class-specific content of almost any image without changing the hidden activations. We identify an insufficiency of the standard cross-entropy loss as a reason for these failures. Further, we extend this objective based on an informationtheoretic analysis so it encourages the model to consider all task-dependent features in its decision. This provides the first approach tailored explicitly to overcome excessive invariance and resulting vulnerabilities.

Out-of-Distribution Generalization via Risk Extrapolation (REx)

“…Improving the generalization of deep learning models has become a major research topic, with many different threads of research including Bayesian deep learning (Neal, 1996;Gal, 2016), adversarial (Engstrom et al, 2019;Jacobsen et al, 2018) and non-adversarial (Hendrycks & Dietterich, 2019;Yin et al, 2019) robustness, causality (Arjovsky et al, 2019), and other works aimed at distinguishing statistical features from semantic features (Gowal et al, 2019;Geirhos et al, 2018). While neural networks often exhibit superhuman generalization performance on the training distribution, they can be extremely sensitive to minute changes in distribution (Su et al, 2019;Engstrom et al, 2017; In this work, we consider out-of-distribution (OoD) generalization, where a model must generalize to new distributions at test time without seeing any training data from them.…”

Guided Image Generation with Conditional Invertible Neural Networks

“…This also contributes to mixing the variables between layers, complementing the soft permutations. Similarly, [18] uses a discrete cosine transform as a final transformation in their INN, to replace global average pooling.…”

Towards an Adversarially Robust Normalization Approach

“…Szegedy et al 2013 linked adversarial vulnerability to blind spots in the discontinuous classification boundary of the neural network, Goodfellow et al 2014 blamed it on the local linearity of neural networks and showed it by constructing an attack that leverages this property. Some recent work has connected it with random noise (Fawzi et al, 2016;Ford et al, 2019), spurious correlations learned by neural networks (Ilyas et al, 2019), insufficient data (Schmidt et al, 2018) high dimensions of input data (Gilmer et al, 2018;Fawzi et al, 2018), and distributional shift (Jacobsen et al, 2018;Ding et al, 2019). Similarly, researchers have also focused on constructing techniques to fight against these attacks.…”

“…While there exist a plethora of reasons for the adversarial behavior of neural networks (Jacobsen et al, 2018;Simon-Gabriel et al, 2018;Yuan et al, 2019;Ilyas et al, 2019;Geirhos et al, 2018), a recent study by Galloway et al 2019 has shown that BatchNorm is one of them. They have empirically shown that we can enhance the robustness of neural networks against adversarial perturbations by removing BatchNorm.…”