On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks

Abstract

While the literature on security attacks and defense of Machine Learning (ML) systems mostly focuses on unrealistic adversarial examples, recent research has raised concern about the under-explored field of realistic adversarial attacks and their implications on the robustness of real-world systems. Our paper paves the way for a better understanding of adversarial robustness against realistic attacks and makes two major contributions. First, we conduct a study on three real-world use cases (text classification, botnet detection, malware detection)) and five datasets in order to evaluate whether unrealistic adversarial examples can be used to protect models against realistic examples. Our results reveal discrepancies across the use cases, where unrealistic examples can either be as effective as the realistic ones or may offer only limited improvement. Second, to explain these results, we analyze the latent representation of the adversarial examples generated with realistic and unrealistic attacks. We shed light on the patterns that discriminate which unrealistic examples can be used for effective hardening. We release our code, datasets and models to support future research in exploring how to reduce the gap between unrealistic and realistic adversarial attacks.

How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks

“…Finally, because the perturbation size is known to impact success rate and human perceptibility of adversarial attacks in other domains (Simonetto et al, 2021;Dyrmishi et al, 2022), we investigate the relationship between the number of altered words and validity/naturalness.…”

Adversarial Robustness in Multi-Task Learning: Promises and Illusions

“…where θ are the parameters of the network, x is the input data, y i is its associated target, L(θ, x, y i ) is the loss function used, and the strength of the attack. Following Goodfellow, other attacks were proposed, first by adding iterations (I-FGSM) (Kurakin, Goodfellow, and Bengio 2016), projections and random restart (PGD) (Madry et al 2017), momentum (MIM) (Dong et al 2018) and constraints (CoEva2) (Ghamizi et al 2020;Dyrmishi et al 2022). These algorithms can be used without any change on a multi-task model if the attacker only focuses on a single task.…”