Live Trojan Attacks on Deep Neural Networks

Costales, Robby; Mao, Chengzhi; Norwitz, Raphael; Kim, Bryan; Yang, Junfeng

doi:10.48550/arxiv.2004.11370

Cited by 2 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6) Post-deployment: Such a backdoor attack occurs after the ML model has been deployed, particularly during the inference phase [76], [77]. Generally, model weights are tampered [78] by fault-injection (e.g., laser, voltage and rowhammer) attacks [76], [79], [80].…”

Section: ) Collaborative Learningmentioning

confidence: 99%

“…Therefore, they can carry out data poisoning or/and model poisoning to backdoor the model returned later to the user who outsources the model training. Given control of the training process, it is worth mentioning that the attacker can always take the evasion-of-the-defense objectives into the loss function to adaptively bypass existing countermeasures [11], [52], [77], [102].…”

Section: A Outsourcing Attackmentioning

confidence: 99%

“…Weight Tamper. As an early work, Dumford et al [78] examine the feasibility of backdooring a face recognition system through solely perturbing weights that are resided in specific layers-a similar study is demonstrated in [77]. Both work [77], [78] intuitively assume that the attacker somehow can access the model resided in the host file storage system or memory, e.g., through a toolkit, and tamper the weight after model deployment.…”

Section: E Post-deployment Attackmentioning

confidence: 99%

“…As an early work, Dumford et al [78] examine the feasibility of backdooring a face recognition system through solely perturbing weights that are resided in specific layers-a similar study is demonstrated in [77]. Both work [77], [78] intuitively assume that the attacker somehow can access the model resided in the host file storage system or memory, e.g., through a toolkit, and tamper the weight after model deployment. It demonstrates the possibility of backdooring a model through perturbing the weights, though it is hugely computationally hungry to achieve a good attack successful rate, e.g., merely up to 50% [78].…”

Section: E Post-deployment Attackmentioning

confidence: 99%

“…The main reason is an extensive iterative search over the weights that will be tampered for the sake of optimizing the ASR [78]. Costales et al [77] takes the evasion defense as an objective when searching for parameters to be modified to bypass existing online inspection defense, specifically, evading the STRIP [82] (STRIP is detailed in section V).…”

Section: E Post-deployment Attackmentioning

confidence: 99%

See 4 more Smart Citations

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Gao,

Doan,

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

Backdoor attacks insert hidden associations or triggers to the deep learning models to override correct inference such as classification and make the system perform maliciously according to the attacker-chosen target while behaving normally in the absence of the trigger. As a new and rapidly evolving realistic attack, it could result in dire consequences, especially considering that the backdoor attack surfaces are broad. In 2019, the U.S. Army Research Office started soliciting countermeasures and launching TrojAI project, the National Institute of Standards and Technology has initialized a corresponding online competition accordingly.However, there is still no systematic and comprehensive review of this emerging area. Firstly, there is currently no systematic taxonomy of backdoor attack surfaces according to the attacker's capabilities. In this context, attacks are diverse and not combed. Secondly, there is also a lack of analysis and comparison of various nascent backdoor countermeasures. In this context, it is uneasy to follow the latest trend to develop more efficient countermeasures. Therefore, this work aims to provide the community with a timely review of backdoor attacks and countermeasures. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdoor inspection, and post backdoor removal. Accordingly, we review countermeasures and compare and analyze their advantages and disadvantages. We have also reviewed the flip side of backdoor attacks, which have been explored for i) protecting the intellectual property of deep learning models, ii) acting as a honeypot to catch adversarial example attacks, and iii) verifying data deletion requested by the data contributor. Overall, the research on the defense side is far behind the attack side, and there is no single defense that can prevent all types of backdoor attacks. In some This version might be updated.

show abstract