TBT: Targeted Neural Network Attack With Bit Trojan

Rakin, Adnan Siraj; He, Zhezhi; Fan, Deliang

doi:10.1109/cvpr42600.2020.01321

Cited by 150 publications

(65 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That technique uses a pre-trained network to learn an attack model that can be directly used to generate images that would fool the victim model. In another example of backdoor attacks, Rakin et al [146] generated a Trojan trigger to locate and flip the vulnerable bits of a DNN in DRAM to make it misbehave. It is noted in [147] that static backdoor attack on images do not work well for videos.…”

Section: Backdoor Attacksmentioning

confidence: 99%

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

2018

View full text Add to dashboard Cite

Deep learning is at the heart of the current rise of artificial intelligence. In the field of Computer Vision, it has become the workhorse for applications ranging from self-driving cars to surveillance and security. Whereas deep neural networks have demonstrated phenomenal success (often beyond human capabilities) in solving complex problems, recent studies show that they are vulnerable to adversarial attacks in the form of subtle perturbations to inputs that lead a model to predict incorrect outputs. For images, such perturbations are often too small to be perceptible, yet they completely fool the deep learning models. Adversarial attacks pose a serious threat to the success of deep learning in practice. This fact has recently lead to a large influx of contributions in this direction. This article presents the first comprehensive survey on adversarial attacks on deep learning in Computer Vision. We review the works that design adversarial attacks, analyze the existence of such attacks and propose defenses against them. To emphasize that adversarial attacks are possible in practical conditions, we separately review the contributions that evaluate adversarial attacks in the real-world scenarios. Finally, drawing on the reviewed literature, we provide a broader outlook of this research direction.

show abstract

Section: Backdoor Attacksmentioning

confidence: 99%

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

2018

View full text Add to dashboard Cite

show abstract

“…For example, when the model training is outsourced to a third party, a malicious third party can insert a backdoor into the returned model [5]. Besides outsourcing, there are other attack surfaces including: training data is from multiple sources that are not fully trusted [13]; usage of pretrained model that contains backdoor could propagate to the downstream model after transfer learning [14]; collaborative learning, e.g., federated learning participants can manipulate their local data as well as updated local model to insert backdoor into the global model [15]; the third party contributed code snippet e.g., packages or modules of a given framework e.g., tensforflow used for training has been manipulated [16]; the model parameters are flipped to implant backdoor even after the model has been deployed in the cloud [17].…”

Section: B Backdoor Attacks On Deep Learningmentioning

confidence: 99%

Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World

Ma¹,

Yin-shan²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning (DL) models have been shown to be vulnerable to recent backdoor attacks. A backdoored model behaves normally for inputs containing no attacker-secretlychosen trigger and maliciously for inputs with the trigger. To date, backdoor attacks and countermeasures mainly focus on classification tasks, in particular, image classification. Most of these backdoor attacks have been implemented in the digital world with digital triggers. Besides the classification tasks, object detection systems are also designed to identify the location of an object, which is a fundamental basis for various computer vision tasks. However, there is no investigation and understanding of the backdoor vulnerability of the object detector, even in the digital world with digital triggers.For the first time, this work demonstrates that existing object detectors are inherently susceptible to physical backdoor attacks, thus revealing severe real-world security threats, e.g., when malicious cloaking is abused. We use a natural T-shirt bought from a market as a trigger to enable the cloaking effect-the person bounding-box disappears in front of the object detector. We show that such a backdoor can be implanted from two widely exploitable attack scenarios into the object detector, which is outsourced or fine-tuned through a pretrained model. We have extensively evaluated three popular object detection algorithms: anchor-based Yolo-V3, Yolo-V4, and anchor-free CenterNet. Building upon 19 videos (about 11,800 frames in total) shot in real-world scenes, we confirm that the backdoor attack is robust against various factors: movement, distance, angle, nonrigid deformation, and lighting. Specifically, the attack success rate (ASR) in most videos is 100% or close to it, while the clean data accuracy of the backdoored model is the same as its clean counterpart. The latter implies that it is infeasible to detect the backdoor behavior merely through a validation set. The averaged ASR still remains sufficiently high to be 78% in the transfer learning attack scenarios evaluated on CenterNet. The comprehensive demo video (up to 5 minutes) is available at https://youtu.be/Q3HOF4OobbY.

show abstract

“…For example, the authors demonstrated that they could embed a łfork bombž in the neural network and execute it on the target system. Similarly, the authors in [43] present an algorithm which generates a trigger that is speciically constructed to ind łvulnerablež bits of the weights. They then perform bit-lip attacks (e.g.…”

Section: Neural Trojansmentioning

confidence: 99%

Diverse, Neural Trojan Resilient Ecosystem of Neural Network IP

Olney

Karam

2022

J. Emerg. Technol. Comput. Syst.

View full text Add to dashboard Cite

Adversarial machine learning is a prominent research area aimed towards exposing and mitigating security vulnerabilities in AI/ML algorithms and their implementations. Data poisoning and neural Trojans enable an attacker to drastically change the behavior and performance of a Convolutional Neural Network (CNN) merely by altering some of the input data during training. Such attacks can be catastrophic in the field, e.g. for self-driving vehicles. In this paper, we propose deploying a CNN as an ecosystem of variants , rather than a singular model. The ecosystem is derived from the original trained model, and though every derived model is structurally different, they are all functionally equivalent to the original and each other. We propose two complementary techniques: stochastic parameter mutation , where the weights θ of the original are shifted by a small, random amount, and a delta-update procedure which functions by XOR’ing all of the parameters with an update file containing the Δθ values. This technique is effective against transferability of a neural Trojan to the greater ecosystem by amplifying the Trojan’s malicious impact to easily detectable levels; thus, deploying a model as an ecosystem can render the ecosystem more resilient against a neural Trojan attack.

show abstract

TBT: Targeted Neural Network Attack With Bit Trojan

Cited by 150 publications

References 18 publications

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World

Diverse, Neural Trojan Resilient Ecosystem of Neural Network IP

Contact Info

Product

Resources

About