De-Pois: An Attack-Agnostic Defense against Data Poisoning Attacks

Chen, Jian; Zhang, Xuxin; Zhang, Rui; Wang, Chen; Ling, Liu

doi:10.1109/tifs.2021.3080522

Cited by 74 publications

(36 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is important to highlight the attacker's capability in other scenarios and the assumptions imposed by the attacker, being sometimes to optimistic while in other scenarios represent more realistic conditions [71]. In the literature, there are many examples related to assumptions in regards to the capability of the attacker.…”

Section: A Discussion Archivesmentioning

confidence: 99%

“…This defense system employs a selfadaptive learning framework to detect and avoid suspicious results falling in the category of false negative to take part of the training process. [71] is an attack-agnostic defense system against poisoning attacks named De-Pois. The defense strategy depicted in this work is not to attack-specific, i.e.…”

Section: Svm Resistance Enhancementmentioning

confidence: 99%

See 1 more Smart Citation

Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

Ramirez¹,

Kim²,

Hamadi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Machine learning models have been widely adopted in several fields. However, most recent studies have shown several vulnerabilities from attacks with a potential to jeopardize the integrity of the model, presenting a new window of research opportunity in terms of cyber-security. This survey is conducted with a main intention of highlighting the most relevant information related to security vulnerabilities in the context of machine learning (ML) classifiers; more specifically, directed towards training procedures against data poisoning attacks, representing a type of attack that consists of tampering the data samples fed to the model during the training phase, leading to a degradation in the model's overall accuracy during the inference phase. This work compiles the most relevant insights and findings found in the latest existing literatures addressing this type of attacks. Moreover, this paper also covers several defense techniques that promise feasible detection and mitigation mechanisms, capable of conferring a certain level of robustness to a target model against an attacker. A thorough assessment is performed on the reviewed works, comparing the effects of data poisoning on a wide range of ML models in real-world conditions, performing quantitative and qualitative analyses. This paper analyzes the main characteristics for each approach including performance success metrics, required hyperparameters, and deployment complexity. Moreover, this paper emphasizes the underlying assumptions and limitations considered by both attackers and defenders along with their intrinsic properties such as: availability, reliability, privacy, accountability, interpretability, etc. Finally, this paper concludes by making references of some of main existing research trends that provide pathways towards future research directions in the field of cyber-security.

show abstract

Section: A Discussion Archivesmentioning

confidence: 99%

Section: Svm Resistance Enhancementmentioning

confidence: 99%

Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

Ramirez¹,

Kim²,

Hamadi³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Muñoz-González et al [27] proposed a similar generative adversarial network based data poisoning method that uses a generator to generate poisoning samples which can maximize classification errors in the target task and meanwhile minimize the discriminator's ability in distinguishing poisoned data from normal data. These data poisoning methods are conducted on centralized datasets, where the attacker has strong prior knowledge of the training dataset [8]. Thus, they are not applicable when training data is decentralized and cannot be exchanged.…”

Section: Related Workmentioning

confidence: 99%

FedAttack: Effective and Covert Poisoning Attack on Federated Recommendation via Hard Sampling

Wu¹,

Wu²,

Qi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a feasible technique to learn personalized recommendation models from decentralized user data. Unfortunately, federated recommender systems are vulnerable to poisoning attacks by malicious clients. Existing recommender system poisoning methods mainly focus on promoting the recommendation chances of target items due to financial incentives. In fact, in real-world scenarios, the attacker may also attempt to degrade the overall performance of recommender systems. However, existing general FL poisoning methods for degrading model performance are either ineffective or not concealed in poisoning federated recommender systems. In this paper, we propose a simple yet effective and covert poisoning attack method on federated recommendation, named FedAttack. Its core idea is using globally hardest samples to subvert model training. More specifically, the malicious clients first infer user embeddings based on local user profiles. Next, they choose the candidate items that are most relevant to the user embeddings as hardest negative samples, and find the candidates farthest from the user embeddings as hardest positive samples. The model gradients inferred from these poisoned samples are then uploaded to the server for aggregation and model update. Since the behaviors of malicious clients are somewhat similar to users with diverse interests, they cannot be effectively distinguished from normal clients by the server. Extensive experiments on two benchmark datasets show that FedAttack can effectively degrade the performance of various federated recommender systems, meanwhile cannot be effectively detected nor defended by many existing methods.

show abstract

“…Toward robustness. In order for the models to defend against the deceptions or threats, researchers have set out to work on closing the gap between adversarial accuracy and the standard accuracy [6,34], and a wide range of defense methods for different types of attack [8,45,56]. Adversarial training, discussed in Section 2.3, is the most popular method against adversarial attacks.…”

Section: Related Workmentioning

confidence: 99%

Learning Representations Robust to Group Shifts and Adversarial Examples

Chiu¹,

Ma²

2022

Preprint

View full text Add to dashboard Cite

Despite the high performance achieved by deep neural networks on various tasks, extensive studies have demonstrated that small tweaks in the input could fail the model predictions. This issue of deep neural networks has led to a number of methods to improve model robustness, including adversarial training and distributionally robust optimization. Though both of these two methods are geared towards learning robust models, they have essentially different motivations: adversarial training attempts to train deep neural networks against perturbations, while distributional robust optimization aims at improving model performance on the most difficult "uncertain distributions". In this work, we propose an algorithm that combines adversarial training and group distribution robust optimization to improve robust representation learning. Experiments on three image benchmark datasets illustrate that the proposed method achieves superior results on robust metrics without sacrificing much of the standard measures.

show abstract

De-Pois: An Attack-Agnostic Defense against Data Poisoning Attacks

Cited by 74 publications

References 14 publications

Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

FedAttack: Effective and Covert Poisoning Attack on Federated Recommendation via Hard Sampling

Learning Representations Robust to Group Shifts and Adversarial Examples

Contact Info

Product

Resources

About