Backdoor Attacks in Federated Learning by Rare Embeddings and Gradient Ensembling

Abstract

Recent advances in federated learning have demonstrated its promising capability to learn on decentralized datasets. However, a considerable amount of work has raised concerns due to the potential risks of adversaries participating in the framework to poison the global model for an adversarial purpose. This paper investigates the feasibility of model poisoning for backdoor attacks through rare word embeddings of NLP models in text classification and sequence-to-sequence tasks. In text classification, less than 1% of adversary clients suffices to manipulate the model output without any drop in the performance of clean sentences. For a less complex dataset, a mere 0.1% of adversary clients is enough to poison the global model effectively. We also propose a technique specialized in the federated learning scheme called gradient ensemble, which enhances the backdoor performance in all experimental settings.

Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions

“…For example, Wan et al [116] found that ASR using the FedAvg algorithm was less than 75% effective with most defenses when one of ten clients was malicious in CV tasks. However, Yoo et al [117] found that ASR was easily more than 95% effective on most attacks with most defenses when one of ten clients was malicious in NLP tasks. One reason for this difference may be that detecting NLP backdoors is more difficult.…”

Federated Learning Based Privacy Ensured Sensor Communication in IoT Networks: A Taxonomy, Threats and Attacks

“…Whereas, when the training rules are manipulated by the adversaries, the attack is known as training rule manipulation attack [22], [91]. Backdoor attacks are performed by adding the stealth backdoors to the global model such that the overall accuracy of the model is retained [93]. In FL-systems, the compromised devices can add the backdoor and train the model on the backdoor data.…”

A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspective

“…Direct boosting [123] Boosting malicious updates Separated boosting [123] Regularized update boosting Model replacement [124] Replace converging global model PGD [125] Bounded update projection Edge case + PGD [46] PGD on minority samples Median interval [59] Median cheating with normalized updates DBA [126] Distributed backdoor trigger TrojanDBA [127] Distributed and learnable trigger Neurotoxin [128] Tampering insignificant model weights RL Neurotoxin [129] Searching Neurotoxin parameters with RL F3BA [130] Sign-flipping on insignificant weights Rare word embedding [131] Tampering stale word embeddings Future update approximation [132] Estimating future updates from malicious clients Sudden collapse [133] Estimating potent malicious gradients Trigger patterns vary from one attack to the other. We summarize existing triggers in Fig.…”

“…Neurotoxin is recently enhanced by authors of [129] who employ RL to find better hyperparameters for the attack. Rare word embedding attack proposed in [131] shares a similar idea with Neurotoxin in the sense that it manipulates word embeddings of rare words as they are not likely to be updated by benign clients. The effectiveness of the rare word embedding attack can be further amplified by the gradient ensembling method [131].…”

“…Rare word embedding attack proposed in [131] shares a similar idea with Neurotoxin in the sense that it manipulates word embeddings of rare words as they are not likely to be updated by benign clients. The effectiveness of the rare word embedding attack can be further amplified by the gradient ensembling method [131]. The attacker intentionally stores the global models from multiple rounds, then gradients of backdoor word embeddings are computed for all these models.…”