Generating Natural Language Adversarial Examples

Abstract

Deep neural networks (DNNs) are vulnerable to adversarial examples, perturbations to correctly classified examples which can cause the model to misclassify. In the image domain, these perturbations are often virtually indistinguishable to human perception, causing humans and state-of-the-art models to disagree. However, in the natural language domain, small perturbations are clearly perceptible, and the replacement of a single word can drastically alter the semantics of the document. Given these challenges, we use a black-box population-based optimization algorithm to generate semantically and syntactically similar adversarial examples that fool well-trained sentiment analysis and textual entailment models with success rates of 97% and 70%, respectively. We additionally demonstrate that 92.3% of the successful sentiment analysis adversarial examples are classified to their original label by 20 human annotators, and that the examples are perceptibly quite similar. Finally, we discuss an attempt to use adversarial training as a defense, but fail to yield improvement, demonstrating the strength and diversity of our adversarial examples. We hope our findings encourage researchers to pursue improving the robustness of DNNs in the natural language domain.

Certified Robustness to Adversarial Word Substitutions

“…Word substitution perturbations. We base our sets of allowed word substitutions S(x, i) on the substitutions allowed by Alzantot et al (2018). They demonstrated that their substitutions lead to adversarial examples that are qualitatively similar to the original input and retain the original label, as judged by humans.…”

“…We make three modifications to this approach. First, in Alzantot et al (2018), the adversary applies substitutions one at a time, and the neighborhoods and language model scores are computed relative to the current altered version of the input. This results in a hard-to-define attack surface, as changing one word can allow or disallow changes to other words.…”

“…1. Genetic attack accuracy: Alzantot et al (2018) demonstrate the effectiveness of a genetic algorithm that searches for perturbationsz that cause model misclassification. The algorithm maintains a "population" of candidatez's and repeatedly perturbs and combines them.…”

Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification

“…Adversarial training is the prevailing counter-measure to build a robust model (Goodfellow et al, 2015;Iyyer et al, 2018;Marzinotto et al, 2019;Cheng et al, 2019; by mixing adversarial examples with the original ones during training the model. However, these adversarial examples can be detected and deactivated by a genetic algorithm (Alzantot et al, 2018). This method also requires retraining, which can be time and cost consuming for large-scale models.…”

Finding Generalizable Evidence by Learning to Convince Q&A Models

“…In machine translation, attention learns to align foreign words with their native counterparts (Bahdanau et al, 2015). On the other hand, neural networks often do not behave as humans (Szegedy et al, 2014;Jia and Liang, 2017;Ribeiro et al, 2018;Alzantot et al, 2018). Convolutional networks rely heavily on texture (Geirhos et al, 2019), while humans rely on shape (Landau et al, 1988).…”