Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification

Abstract

Deep neural networks (DNNs) have transformed several arti cial intelligence research areas including computer vision, speech recognition, and natural language processing. However, recent studies demonstrated that DNNs are vulnerable to adversarial manipulations at testing time. Speci cally, suppose we have a testing example, whose label can be correctly predicted by a DNN classi er. An a acker can add a small carefully cra ed noise to the testing example such that the DNN classi er predicts an incorrect label, where the cra ed testing example is called adversarial example. Such a acks are called evasion a acks. Evasion a acks are one of the biggest challenges for deploying DNNs in safety and security critical applications such as self-driving cars. In this work, we develop new methods to defend against evasion a acks. Our key observation is that adversarial examples are close to the classi cation boundary. erefore, we propose region-based classi cation to be robust to adversarial examples. For a benign/adversarial testing example, we ensemble information in a hypercube centered at the example to predict its label. Speci cally, we sample some data points from the hypercube centered at the testing example in the input space; we use an existing DNN to predict the label for each sampled data point; and we take a majority vote among the labels of the sampled data points as the label for the testing example. In contrast, traditional classi ers are point-based classi cation, i.e., given a testing example, the classi er predicts its label based on the testing example alone. Our evaluation results on MNIST and CIFAR-10 datasets demonstrate that our region-based classi cation can signi cantly mitigate evasion a acks without sacri cing classi cation accuracy on benign examples. Speci cally, our region-based classi cation achieves the same classi cation accuracy on testing benign examples as point-based classi cation, but our region-based classi cation is signi cantly more robust than point-based classi cation to various evasion a acks.

Certified Robustness of Community Detection against Adversarial Structural Perturbation via Randomized Smoothing

“…Randomized smoothing was initially proposed as empirical defenses [6,26] without formal certified robustness guarantees. For example, Cao & Gong [6] proposed to use uniform noise sampled from a hypercube centered at an input. Lecuyer et al [20] derived the first certified robustness guarantee for randomized smoothing with Gaussian or Laplacian noise by utilizing differential privacy techniques.…”

Robustness of Neural Networks to Parameter Quantization

“…We then checked whether the input labeled will be flipped in the perturbed network using Eq. (8). In the figures, green and red points represent non-flippable and flippable inputs, respectively.…”

Feedback Learning for Improving the Robustness of Neural Networks

“…Cao and Gong aimed to increase the robustness of neural networks [30]. Instead of using only a test sample to determine which class it belongs to, hundreds of neighboring samples are generated in the surrounding hypercube and a voting algorithm is applied to decide the true label of the test sample.…”

“…Inspired by this region-based classification method [30], He et al [23] moved from hypercubes to larger neighborhoods. They proposed an orthogonal-direction ensemble attack called OptMargin, which could evade the region-based classification defense mentioned in [30].…”

“…Inspired by this region-based classification method [30], He et al [23] moved from hypercubes to larger neighborhoods. They proposed an orthogonal-direction ensemble attack called OptMargin, which could evade the region-based classification defense mentioned in [30]. They also analyzed margins (between samples and decision boundaries) and adjacent class information, then built a simple neural network to detect adversarial examples with these analysis results.…”