A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning

Abstract

The prosperity of machine learning has been accompanied by increasing attacks on the training process. Among them, poisoning attacks have become an emerging threat during model training. Poisoning attacks have profound impacts on the target models, e.g., making them unable to converge or manipulating their prediction results. Moreover, the rapid development of recent distributed learning frameworks, especially federated learning, has further stimulated the development of poisoning attacks. Defending against poisoning attacks is challenging and urgent. However, the systematic review from a unified perspective remains blank. This survey provides an in-depth and up-to-date overview of poisoning attacks and corresponding countermeasures in both centralized and federated learning. We firstly categorize attack methods based on their goals. Secondly, we offer detailed analysis of the differences and connections among the attack techniques. Furthermore, we present countermeasures in different learning framework and highlight their advantages and disadvantages. Finally, we discuss the reasons for the feasibility of poisoning attacks and address the potential research directions from attacks and defenses perspectives, separately.

HyFL: A Hybrid Approach For Private Federated Learning

“…Malicious FL participants can try to manipulate the global model to either produce specific outputs for specific inputs or simply degrade the overall accuracy. These attacks are referred to as backdoor (Bagdasaryan et al, 2020;Xie et al, 2020) and poisoning attacks (Tian et al, 2022), respectively. In terms of poisoning attacks, the two options are to perform data poisoning (Biggio et al, 2012;Tolpegin et al, 2020) or model poisoning (Wang et al, 2020;Fang et al, 2020).…”

Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions

“…Their survey, however, falls short of assessing or demonstrating the connection between these attacks, as well as the connection between backdoor attacks and defenses. In Backdoor attacks and defenses [37] 2022 Backdoor attacks and defenses in FL [38] 2022 Poisoning attacks and countermeasures [39] 2022 FL challenges, contributions, and trends [40] 2021 Privacy-preserving FL [41] 2021…”

When Machine Learning Models Leak: An Exploration of Synthetic Training Data

“…Privacy attacks in machine learning [5,22] include membership inference attacks [24], model reconstruction attacks such as attribute inference [29], model inversion attacks [11,10], and model extraction attacks [28]. Here, we focus on a form of model inversion attacks, namely, attribute inference attack.…”

“…Model inversion attacks try to recover sensitive features or the full data sample based on output labels and partial knowledge (subset of data) of some features [22,19]. [19] provided a summary of possible assumptions about adversary capabilities and resources for different model inversion attribute inference attacks.…”