Backdoor Attacks Against Deep Learning Systems in the Physical World

Wenger, Emily; Passananti, Josephine; Bhagoji, Arjun Nitin; Yao, Yuanshun; Zheng, Haitao; Zhao, Ben Y.

doi:10.48550/arxiv.2006.14580

Cited by 7 publications

(18 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, previous backdoor attacks usually uses digital triggers rather than natural physical triggers. Only a few recent works [18]- [20] consider the usage of natural triggers and of their focus has been on image classification task. Therefore, there is a lack of study on the efficiency of backdoor attack in real physical world used object detectors.…”

Section: B Backdoor Attacks On Deep Learningmentioning

confidence: 99%

Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World

Ma¹,

Yin-shan²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning (DL) models have been shown to be vulnerable to recent backdoor attacks. A backdoored model behaves normally for inputs containing no attacker-secretlychosen trigger and maliciously for inputs with the trigger. To date, backdoor attacks and countermeasures mainly focus on classification tasks, in particular, image classification. Most of these backdoor attacks have been implemented in the digital world with digital triggers. Besides the classification tasks, object detection systems are also designed to identify the location of an object, which is a fundamental basis for various computer vision tasks. However, there is no investigation and understanding of the backdoor vulnerability of the object detector, even in the digital world with digital triggers.For the first time, this work demonstrates that existing object detectors are inherently susceptible to physical backdoor attacks, thus revealing severe real-world security threats, e.g., when malicious cloaking is abused. We use a natural T-shirt bought from a market as a trigger to enable the cloaking effect-the person bounding-box disappears in front of the object detector. We show that such a backdoor can be implanted from two widely exploitable attack scenarios into the object detector, which is outsourced or fine-tuned through a pretrained model. We have extensively evaluated three popular object detection algorithms: anchor-based Yolo-V3, Yolo-V4, and anchor-free CenterNet. Building upon 19 videos (about 11,800 frames in total) shot in real-world scenes, we confirm that the backdoor attack is robust against various factors: movement, distance, angle, nonrigid deformation, and lighting. Specifically, the attack success rate (ASR) in most videos is 100% or close to it, while the clean data accuracy of the backdoored model is the same as its clean counterpart. The latter implies that it is infeasible to detect the backdoor behavior merely through a validation set. The averaged ASR still remains sufficiently high to be 78% in the transfer learning attack scenarios evaluated on CenterNet. The comprehensive demo video (up to 5 minutes) is available at https://youtu.be/Q3HOF4OobbY.

show abstract

Section: B Backdoor Attacks On Deep Learningmentioning

confidence: 99%

Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World

Ma¹,

Yin-shan²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Backdoor attacks in real physical world. To the best of the authors' knowledge, there is only one work conducted on the physical backdoor attacks [13] (an unpublished work in arXiv). Wenger et al [13] collect 9 different facial accessories in the real world as backdoor triggers, and evaluate the attack effectiveness of these backdoors on the face recognition models.…”

Section: Related Workmentioning

confidence: 99%

“…To the best of the authors' knowledge, there is only one work conducted on the physical backdoor attacks [13] (an unpublished work in arXiv). Wenger et al [13] collect 9 different facial accessories in the real world as backdoor triggers, and evaluate the attack effectiveness of these backdoors on the face recognition models. However, their attacks are carried out under the ideal physical conditions, where the attacker is facing the camera with a proper distance.…”

Section: Related Workmentioning

confidence: 99%

“…The VGGFace is a 16-layer standard face recognition model, which consists of 13 convolutional layers and 3 fully connected layers. We adopt the experimental settings in these exisiting works [6], [13] and download the VGGFace model that pre-trained on the ImageNet [26]. The last three layers of the VGGFace model are fine-turned, and the sofatmax activation layer is replaced, so as to train the VGGFace model on our experimental dataset.…”

Section: A Experimental Setupmentioning

confidence: 99%

“…However, almost all of the existing backdoor attacks are conducted in the digital domain, while few studies have been studied in real physical world. To the best of the authors' knowledge, there is only one unpublished work [13] (a preprint in arXiv) explores the physical backdoor attacks. The authors collects 9 different triggers (e.g., headbands, earrings, etc.)…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Robust Backdoor Attacks against Deep Neural Networks in Real Physical World

Xue,

He,

Sun

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNN) have been widely deployed in various practical applications. However, many researches indicated that DNN is vulnerable to backdoor attacks. The attacker can create a hidden backdoor in target DNN model, and trigger the malicious behaviors by submitting specific backdoor instance. However, almost all the existing backdoor works focused on the digital domain, while few studies investigate the backdoor attacks in real physical world. Restricted to a variety of physical constrains, the performance of backdoor attacks in the real world will be severely degraded. In this paper, we propose a robust physical backdoor attack method, PTB (physical transformations for backdoors), to implement the backdoor attacks against deep learning models in the physical world. Specifically, in the training phase, we perform a series of physical transformations on these injected backdoor instances at each round of model training, so as to simulate various transformations that a backdoor may experience in real world, thus improves its physical robustness. Experimental results on the state-of-the-art face recognition model show that, compared with the methods that without PTB, the proposed attack method can significantly improve the performance of backdoor attacks in real physical world. Under various complex physical conditions, by injecting only a very small ratio (0.5%) of backdoor instances, the success rate of physical backdoor attacks with the PTB method on VGGFace is 82%, while the attack success rate of backdoor attacks without the proposed PTB method is lower than 11%. Meanwhile, the normal performance of target DNN model has not been affected. This paper is the first work on the robustness of physical backdoor attacks, and is hopeful for providing guideline for the subsequent physical backdoor works.

show abstract

Local and Central Differential Privacy for Robustness and Privacy in Federated Learning

Naseri¹,

Hayes²,

Cristofaro³

2022

Proceedings 2022 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

Federated Learning (FL) allows multiple participants to train machine learning models collaboratively by keeping their datasets local while only exchanging model updates. Alas, this is not necessarily free from privacy and robustness vulnerabilities, e.g., via membership, property, and backdoor attacks. This paper investigates whether and to what extent one can use differential Privacy (DP) to protect both privacy and robustness in FL. To this end, we present a first-of-its-kind evaluation of Local and Central Differential Privacy (LDP/CDP) techniques in FL, assessing their feasibility and effectiveness.Our experiments show that both DP variants do defend against backdoor attacks, albeit with varying levels of protectionutility trade-offs, but anyway more effectively than other robustness defenses. DP also mitigates white-box membership inference attacks in FL, and our work is the first to show it empirically. Neither LDP nor CDP, however, defend against property inference. Overall, our work provides a comprehensive, re-usable measurement methodology to quantify the trade-offs between robustness/privacy and utility in differentially private FL.

show abstract

Backdoor Attacks Against Deep Learning Systems in the Physical World

Cited by 7 publications

References 39 publications

Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World

Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World

Robust Backdoor Attacks against Deep Neural Networks in Real Physical World

Local and Central Differential Privacy for Robustness and Privacy in Federated Learning

Contact Info

Product

Resources

About