Reevaluating Adversarial Examples in Natural Language

Abstract

State-of-the-art attacks on NLP models lack a shared definition of what constitutes a successful attack. These differences make the attacks difficult to compare and hindered the use of adversarial examples to understand and improve NLP models. We distill ideas from past work into a unified framework: a successful natural language adversarial example is a perturbation that fools the model and follows four proposed linguistic constraints. We categorize previous attacks based on these constraints. For each constraint, we suggest options for human and automatic evaluation methods. We use these methods to evaluate two state-of-the-art synonym substitution attacks. We find that perturbations often do not preserve semantics, and 38% introduce grammatical errors. Next, we conduct human studies to find a threshold for each evaluation method that aligns with human judgment. Human surveys reveal that to successfully preserve semantics, we need to significantly increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences. With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points. 1

Shortcutted Commonsense: Data Spuriousness in Deep Learning of Commonsense Reasoning

“…A study on algorithms to generate adversarial examples found that they may not fully preserve semantics and may introduce up to 38% of grammatical errors (Morris et al, 2020a). To mitigate this, through comparison with human performance, its authors suggest a number of TextFooler's hyperparameters.…”

Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples

“…With a looser definition, the search space includes more candidate adversarial examples. The more candidates there are, the more likely the search is to find an example that fools the victim model -thereby achieving a higher attack success rate (Morris et al, 2020b).…”

“…Constraints: Morris et al (2020b) proposed a set of linguistic constraints to enforce that x and perturbed x 0 should be similar in both meaning and fluency to make x 0 a valid potential adversarial example. This indicates that the search space should ensure x and x 0 are close in semantic embedding space.…”

“…2.1 Components of an NLP Attack Morris et al (2020b) formulated the process of generating natural language adversarial examples as a system of four components: a goal function, a set of constraints, a transformation, and a search algorithm.…”

A Closer Look into the Robustness of Neural Dependency Parsers Using Better Adversarial Examples

“…However, recent work has shown that their performance can be easily undermined with adversarial examples that would pose no confusion for humans . As an increasing number of successful adversarial attackers have been developed for NLP tasks, the quality of the adversarial examples they generate has been questioned (Morris et al, 2020).…”