Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers

Truong, Loc Dong; Jones, Chace; Hutchinson, Brian; August, Andrew; Praggastis, Brenda; Jasper, Robert; Nichols, Nicole L.; Tuor, Aaron

doi:10.48550/arxiv.2004.11514

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As we limit our data poisoning to a small number of training examples, we believe such techniques would have difficulties detecting poisoning examples. Furthermore, some more recent defenses suggested retraining [33], which we consider to be a good choice against our attack. Still, since such works commonly assume an unlimited amount of clean data, such approaches are not practical.…”

Section: Discussionmentioning

confidence: 99%

Can You Hear It? Backdoor Attacks via Ultrasonic Triggers

Koffas¹,

Xu²,

Conti³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep neural networks represent a powerful option for many realworld applications due to their ability to model even complex data relations. However, such neural networks can also be prohibitively expensive to train, making it common to either outsource the training process to third parties or use pretrained neural networks. Unfortunately, such practices make neural networks vulnerable to various attacks, where one attack is the backdoor attack. In such an attack, the third party training the model may maliciously inject hidden behaviors into the model. Still, if a particular input (called trigger) is fed into a neural network, the network will respond with a wrong result.In this work, we explore the option of backdoor attacks to automatic speech recognition systems where we inject inaudible triggers. By doing so, we make the backdoor attack challenging to detect for legitimate users, and thus, potentially more dangerous. We conduct experiments on two versions of datasets and three neural networks and explore the performance of our attack concerning the duration, position, and type of the trigger. Our results indicate that less than 1% of poisoned data is sufficient to deploy a backdoor attack and reach a 100% attack success rate. What is more, while the trigger is inaudible, making it without limitations with respect to the duration of the signal, we observed that even short, non-continuous triggers result in highly successful attacks. CCS CONCEPTS• Security and privacy → Systems security; • Computing methodologies → Speech recognition; Neural networks.

show abstract

Section: Discussionmentioning

confidence: 99%

Can You Hear It? Backdoor Attacks via Ultrasonic Triggers

Koffas¹,

Xu²,

Conti³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Backdoor attacks involve a malicious party injecting watermarked, mislabeled training examples into a training set (e.g. [13], [29], [9], [30], [27], [17]). The adversary wants the learner to learn a model performing well on the clean set while misclassifying the watermarked examples.…”

Section: Introductionmentioning

confidence: 99%

Excess Capacity and Backdoor Poisoning

Manoj¹,

Blum²

2021

Preprint

View full text Add to dashboard Cite

A backdoor data poisoning attack is an adversarial attack wherein the attacker injects several watermarked, mislabeled training examples into a training set. The watermark does not impact the test-time performance of the model on typical data; however, the model reliably errs on watermarked examples.To gain a better foundational understanding of backdoor data poisoning attacks, we present a formal theoretical framework within which one can discuss backdoor data poisoning attacks for classification problems. We then use this to analyze important statistical and computational issues surrounding these attacks.On the statistical front, we identify a parameter we call the memorization capacity that captures the intrinsic vulnerability of a learning problem to a backdoor attack. This allows us to argue about the robustness of several natural learning problems to backdoor attacks. Our results favoring the attacker involve presenting explicit constructions of backdoor attacks, and our robustness results show that some natural problem settings cannot yield successful backdoor attacks.From a computational standpoint, we show that under certain assumptions, adversarial training can detect the presence of backdoors in a training set. We then show that under similar assumptions, two closely related problems we call backdoor filtering and robust generalization are nearly equivalent. This implies that it is both asymptotically necessary and sufficient to design algorithms that can identify watermarked examples in the training set in order to obtain a learning algorithm that both generalizes well to unseen data and is robust to backdoors.

show abstract

Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers

Cited by 2 publications

References 0 publications

Can You Hear It? Backdoor Attacks via Ultrasonic Triggers

Can You Hear It? Backdoor Attacks via Ultrasonic Triggers

Excess Capacity and Backdoor Poisoning

Contact Info

Product

Resources

About