Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Abstract

Backdoor attacks insert hidden associations or triggers to the deep learning models to override correct inference such as classification and make the system perform maliciously according to the attacker-chosen target while behaving normally in the absence of the trigger. As a new and rapidly evolving realistic attack, it could result in dire consequences, especially considering that the backdoor attack surfaces are broad. In 2019, the U.S. Army Research Office started soliciting countermeasures and launching TrojAI project, the National Institute of Standards and Technology has initialized a corresponding online competition accordingly.However, there is still no systematic and comprehensive review of this emerging area. Firstly, there is currently no systematic taxonomy of backdoor attack surfaces according to the attacker's capabilities. In this context, attacks are diverse and not combed. Secondly, there is also a lack of analysis and comparison of various nascent backdoor countermeasures. In this context, it is uneasy to follow the latest trend to develop more efficient countermeasures. Therefore, this work aims to provide the community with a timely review of backdoor attacks and countermeasures. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdoor inspection, and post backdoor removal. Accordingly, we review countermeasures and compare and analyze their advantages and disadvantages. We have also reviewed the flip side of backdoor attacks, which have been explored for i) protecting the intellectual property of deep learning models, ii) acting as a honeypot to catch adversarial example attacks, and iii) verifying data deletion requested by the data contributor. Overall, the research on the defense side is far behind the attack side, and there is no single defense that can prevent all types of backdoor attacks. In some This version might be updated.

Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World

“…Therefore, this work mainly leverages the data poison to insert a backdoor into the object detector. This attacking strategy has been shown to be very effective to implant backdoor into various models [7]. Opposed to change the label to the targeted class when backdooring a classification model [5], here we modify the annotation of the Poisoning the Training Dataset.…”

“…As this is the first work that investigates the backdoor vulnerability of the object detectors built up DL, there is so far no existing solution to such attack. As a matter of fact, to the best of our knowledge, nearly all (if not all) of the DL backdoor countermeasures are focusing on the classification tasks, especially image classifications [7]. They (i.e., state-ofthe-arts of [39]- [42]) are not immediately mountable on protecting object detection task, which is beyond the classification task.…”

“…Therefore, it is challenging to craft adversarial example patch to be robust in diverse real-world scenes. Distinct from the adversarial example attack, recently, there is a new backdoor attack being revealed, with nearly all studies are on classification tasks, especially image classifications [5]- [7]. A backdoored model behaves normally given normal inputs containing the attacker secretly-chosen trigger, but misbehaves as the attacker intends once the trigger is presented in the input.…”

Robust and Privacy-Preserving Collaborative Learning: A Comprehensive Survey

“…To avoid or mitigate the effects of backdoor attacks on collaborative learning systems, several backdoor defense methods have been proposed [74], [130]- [132]. We divide existing methods into two categories based on the subject of inspection: data and the model inspection.…”

“…Data inspection defenses try to distinguish poisoned data from normal ones, while the model inspection approach [48], [49] relies on anomaly technique to distinguish abnormal behaviour of the models caused by backdoors [130]. These defenses can be carried out during or after the training processing.…”