FABA: An Algorithm for Fast Aggregation against Byzantine Attacks in Distributed Neural Networks

Xia, Qi; Tao, Zeyi; Hao, Zijiang; Li, Qun

doi:10.24963/ijcai.2019/670

Cited by 56 publications

(21 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A trusted execution environment (TEE) is a secure execution environment (mostly connected hardware components) which guarantees code and data loaded inside to be protected with respect to confidentiality and integrity [17]. Although dedicated TEEs for AI systems have been widely studied [87]- [90], the mechanisms for constructing secured environments are broader than just AI dedicated ones. Non-AI system dedicated defense mechanisms could be applied to protect trained machine learning models as the TEE in the testing phase [91].…”

Section: B Defense Mechanisms For Testing Nodesmentioning

confidence: 99%

Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

Ramirez¹,

Kim²,

Hamadi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Machine learning models have been widely adopted in several fields. However, most recent studies have shown several vulnerabilities from attacks with a potential to jeopardize the integrity of the model, presenting a new window of research opportunity in terms of cyber-security. This survey is conducted with a main intention of highlighting the most relevant information related to security vulnerabilities in the context of machine learning (ML) classifiers; more specifically, directed towards training procedures against data poisoning attacks, representing a type of attack that consists of tampering the data samples fed to the model during the training phase, leading to a degradation in the model's overall accuracy during the inference phase. This work compiles the most relevant insights and findings found in the latest existing literatures addressing this type of attacks. Moreover, this paper also covers several defense techniques that promise feasible detection and mitigation mechanisms, capable of conferring a certain level of robustness to a target model against an attacker. A thorough assessment is performed on the reviewed works, comparing the effects of data poisoning on a wide range of ML models in real-world conditions, performing quantitative and qualitative analyses. This paper analyzes the main characteristics for each approach including performance success metrics, required hyperparameters, and deployment complexity. Moreover, this paper emphasizes the underlying assumptions and limitations considered by both attackers and defenders along with their intrinsic properties such as: availability, reliability, privacy, accountability, interpretability, etc. Finally, this paper concludes by making references of some of main existing research trends that provide pathways towards future research directions in the field of cyber-security.

show abstract

Section: B Defense Mechanisms For Testing Nodesmentioning

confidence: 99%

Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

Ramirez¹,

Kim²,

Hamadi³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…1 Label flipping: The attacker ''flips'' the labels of its training data to arbitrary ones (e.g., via a permutation function). 2 Adding noise: An attacker contaminates the dataset by adding noises to degrade the quality of models. 3 Backdoor trigger: An attacker injects a trigger into a small area of the original dataset to cause the classifier misclassifying into the target category.…”

Section: B Byzantine Attackmentioning

confidence: 99%

“…There are two ways to modify the parameters: 1 Modifying the direction and size of the parameter learned from the local dataset, e.g., flipping the signs of local iterates and gradients, or enlarging the magnitudes. 2 Modifying the parameter directly, e.g., randomly sampling a number from the Gaussian distribution and treating it as one of the parameters of the local model.…”

Section: B Byzantine Attackmentioning

confidence: 99%

“…In Krum, the central server chooses only one update that is closest to its neighbors to update the global model, and discards all the other updates, whereas Multi-Krum chooses multiple updates and computes the mean to update the global model. Similar to Multi-Krum, FABA [2] aims to remove the outliers in the uploaded gradients by discarding the gradients that are far away from the mean gradient. However, both Multi-Krum and FABA need to know the number of malicious clients in advance, which makes them difficult to be applied to practical applications.…”

Section: A the Distance Based Defense Schemesmentioning

confidence: 99%

“…Blanchard et al [1] have shown that just one baleful user can compromise the convergence of the training and damage the performance of the ultimate global model. To address this issue, a mounting number of defense strategies against byzantine attacks have been proposed to further safeguard FL [2]-- [6]. Although these research efforts have demonstrated their preliminary success in defeating byzantine attacks, we emphasize that it is still far from practice to provide a full protection for FL.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Challenges and approaches for mitigating byzantine attacks in federated learning

Hu¹,

Lu²,

Wan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently emerged federated learning (FL) is an attractive distributed learning framework in which numerous wireless end-user devices can train a global model with the data remained autochthonous. Compared with the traditional machine learning framework that collects user data for centralized storage, which brings huge communication burden and concerns about data privacy, this approach can not only save the network bandwidth but also protect the data privacy. Despite the promising prospect, byzantine attack, an intractable threat in conventional distributed network, is discovered to be rather efficacious against FL as well. In this paper, we conduct a comprehensive investigation of the state-of-the-art strategies for defending against byzantine attacks in FL. We first provide a taxonomy for the existing defense solutions according to the techniques they used, followed by an across-the-board comparison and discussion. Then we propose a new byzantine attack method called weight attack to defeat those defense schemes, and conduct experiments to demonstrate its threat. The results show that existing defense solutions, although abundant, are still far from fully protecting FL. Finally, we indicate possible countermeasures for weight attack, and highlight several challenges and future research directions for mitigating byzantine attacks in FL.

show abstract

Privacy preserving and secure robust federated learning: A survey

Han,

Lu,

Wang

et al. 2024

Concurrency and Computation

View full text Add to dashboard Cite

SummaryFederated learning (FL) has emerged as a promising solution to address the challenges posed by data silos and the need for global data fusion. It offers a distributed machine learning framework with privacy‐preserving features, allowing model training without the need to collect user data. However, FL also presents significant security and privacy threats that hinder its widespread adoption. The requirements of privacy and security in FL are inherently conflicting. Privacy necessitates the concealment of individual client updates, while security requires the disclosure of client updates to detect anomalies. While most existing research focused on the privacy and security aspects of FL, very few studies have addressed the compatibility of these two demands. In this work, we aim to bridge this gap by proposing a comprehensive defense scheme that ensures privacy, security, and compatibility in FL. We categorize the existing literature into two key directions: privacy defense and security defense. Privacy defense includes methods based on additive masks, differential privacy, homomorphic encryption, and trusted execution environment, whereas security defense encompasses distance‐, performance‐, clustering‐, and similarity‐based anomaly detection techniques and statistical information‐based anomaly update bypassing techniques when the server is trusted and privacy‐compatible anomaly update detection techniques when the server is not trusted. In addition, this article presents decentralized FL solutions based on blockchain. For each direction, we discuss specific technical solutions, their advantages, and disadvantages. By evaluating various defense methods, we identify the most suitable approach to address the primary challenge of “achieving a secure and robust FL system against malicious adversaries while protecting users' privacy.” We then propose a theoretical reference framework for end‐to‐end protection of privacy and security in FL for the key problem, which summarizes the attack surface of FL systems from the client to the server under the security model where the client and server are malicious. Leveraging the strengths and characteristics of existing schemes, our proposed framework integrates multiple techniques to strike a balance between privacy, usability, and efficiency. This framework serves as a valuable reference and provides insights for future work in the field. Finally, we also provide recommendations for future research directions in this field.

show abstract

FABA: An Algorithm for Fast Aggregation against Byzantine Attacks in Distributed Neural Networks

Cited by 56 publications

References 6 publications

Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

Challenges and approaches for mitigating byzantine attacks in federated learning

Privacy preserving and secure robust federated learning: A survey

Contact Info

Product

Resources

About