Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

Goldblum, Micah; Tsipras, Dimitris; Xie, Chulin; Schwarzschild, Avi; Song, Dawn; Mądry, Aleksander; Li, Bo; Goldstein, Tom

doi:10.48550/arxiv.2012.10544

Cited by 16 publications

(22 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Solely altering the dataset, i.e., a Backdoor without Boosting is considered a Data Poisoning-based Backdoor. However, when using Boosting techniques, Backdoors are considered Model Poisoning-based Backdoors [74].…”

Section: A Targeting Integrity and Availabilitymentioning

confidence: 99%

“…However, Server Cleaning requires a separated aggregator algorithm. For such purposes, the server might leverage different techniques based on the intrinsic properties of the updates [74].…”

Section: B Defending Integrity and Availabilitymentioning

confidence: 99%

“…Other authors [15] further studied privacy implications in FL by proposing a taxonomy for privacy-preserving defenses. Regarding security, authors in [74] analyzed the security implications of datasets.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

On the Security & Privacy in Federated Learning

Abad¹,

Picek²,

Ramírez-Durán³

et al. 2021

Preprint

View full text Add to dashboard Cite

Advances in Machine Learning (ML) and its wide range of applications boosted its popularity. Recent privacy awareness initiatives as the EU General Data Protection Regulation (GDPR) -European Parliament and Council Regulation No 2016/679, subdued ML to privacy and security assessments. Federated Learning (FL) grants a privacy-driven, decentralized training scheme that improves ML models' security. The industry's fast-growing adaptation and security evaluations of FL technology exposed various vulnerabilities. Depending on the FL phase, i.e., training or inference, the adversarial actor capabilities, and the attack type threaten FL's confidentiality, integrity, or availability (CIA). Therefore, the researchers apply the knowledge from distinct domains as countermeasures, like cryptography and statistics.This work assesses the CIA of FL by reviewing the state-of-theart (SoTA) for creating a threat model that embraces the attack's surface, adversarial actors, capabilities, and goals. We propose the first unifying taxonomy for attacks and defenses by applying this model. Additionally, we provide critical insights extracted by applying the suggested novel taxonomies to the SoTA, yielding promising future research directions.

show abstract

Section: A Targeting Integrity and Availabilitymentioning

confidence: 99%

Section: B Defending Integrity and Availabilitymentioning

confidence: 99%

See 1 more Smart Citation

On the Security & Privacy in Federated Learning

Abad¹,

Picek²,

Ramírez-Durán³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Backdoor Attacks on Neural Networks. The key idea of backdoor attacks [12,21,23,53] is to inject hidden behaviors into a model, such that a test-time input stamped with a specific backdoor trigger (e.g. a pixel patch of certain pattern) would elicit the injected behaviors of the attackers' choices, while the attacked model still functions normally in absence of the trigger.…”

Section: Related Workmentioning

confidence: 99%

“…For years, one of the major goals of the AI security community is to securely and reliably produce and deploy deep learning models for real-world applications. To this end, data poisoning based backdoor attacks [12,21,53,72] on deep neural networks (DNNs) in the production stage (or training stage) and corresponding defenses [11,13,74] are extensively explored in recent years.…”

Section: Introductionmentioning

confidence: 99%

Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks

Xie

Pan

et al. 2021

Preprint

View full text Add to dashboard Cite

One major goal of the AI security community is to securely and reliably produce and deploy deep learning models for real-world applications. To this end, data poisoning based backdoor attacks on deep neural networks (DNNs) in the production stage (or training stage) and corresponding defenses are extensively explored in recent years. Ironically, backdoor attacks in the deployment stage, which can often happen in unprofessional users' devices and are thus arguably far more threatening in real-world scenarios, draw much less attention of the community. We attribute this imbalance of vigilance to the weak practicality of existing deployment-stage backdoor attack algorithms and the insufficiency of real-world attack demonstrations. To fill the blank, in this work, we study the realistic threat of deployment-stage backdoor attacks on DNNs. We base our study on a commonly used deployment-stage attack paradigm -adversarial weight attack, where adversaries selectively modify model weights to embed backdoor into deployed DNNs. To approach realistic practicality, we propose the first gray-box and physically realizable weights attack algorithm for backdoor injection, namely subnet replacement attack (SRA), which only requires architecture information of the victim model and can support physical triggers in the real world. Extensive experimental simulations and system-level real-world attack demonstrations are conducted. Our results not only suggest the effectiveness and practicality of the proposed attack algorithm, but also reveal the practical risk of a novel type of computer virus that may widely spread and stealthily inject backdoor into DNN models in user devices. By our study, we call for more attention to the vulnerability of DNNs in the deployment stage.

show abstract

Using Random Perturbations to Mitigate Adversarial Attacks on NLP Models

Swenor

2022

AAAI

View full text Add to dashboard Cite

Deep learning models have excelled in solving many problems in Natural Language Processing, but are susceptible to extensive vulnerabilities. We offer a solution to this vulnerability by using random perturbations such as spelling correction, synonym substitution, or dropping the word. These perturbations are applied to random words in random sentences to defend NLP models against adversarial attacks. Our defense methods are successful in returning attacked models to their original accuracy within statistical significance.

show abstract

Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

Cited by 16 publications

References 55 publications

On the Security & Privacy in Federated Learning

On the Security & Privacy in Federated Learning

Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks

Using Random Perturbations to Mitigate Adversarial Attacks on NLP Models

Contact Info

Product

Resources

About