Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures

Bagdasaryan, Eugene; Shmatikov, Vitaly

doi:10.1109/sp46214.2022.9833572

Cited by 62 publications

(78 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our experiment, this paper uses PyTorch's built-in distribution package 24 , and uses synchronous logic to implement the server's parameter update process. In this section, we compare the defense effects of different defense models and analyze the possible problems.…”

Section: Experiments and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Aggregation algorithm based on consensus verification

Qin

2023

Preprint

View full text Add to dashboard Cite

Distributed learning, as the most popular solution for training large-scale data for deep learning, consists of multiple participants collaborating on data training tasks. However, the malicious behavior of some during the training process, like Byzantine participants who would interrupt or control the learning process, will trigger the crisis of data security. Although recent existing defense mechanisms use the variability of Byzantine node gradients to clear Byzantine values, it is still unable to identify and then clear the delicate disturbance/attack. To address this critical issue, we propose an algorithm named consensus aggregation in this paper. This algorithm allows computational nodes to use the information of verification nodes to verify the effectiveness of the gradient in the perturbation attack, reaching a consensus based on the effective verification of the gradient. Then the server node uses the gradient as the valid gradient for gradient aggregation calculation through the consensus reached by other computing nodes. The proposed algorithm outperforms existing common aggregation algorithms (Krum, Trimmed Mean, and Bulyan) in the face of delicate disturbances and attacks on the MNIST and CIFAR10 datasets. In our experiments, we show that the proposed consensus aggregation algorithm successfully defends against a variety of Byzantine problems and effectively solves delicate disturbance/attack in Byzantine problems in distributed learning.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

“…In 2018, Bagdasaryan et al 24 . demonstrated a backdoor attack against federated learning that optimized the attacker to a model with a backdoor and added an item in the loss to keep the new parameter close to the original parameter to achieve an efficient attack.…”

Section: Existing Attacksmentioning

confidence: 99%

Aggregation algorithm based on consensus verification

Qin

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Specifically, a trojaned model behaves normally on benign inputs but has attacker-chosen behaviors if the input contains the trigger. In this paper, we focus on classification models following previous papers on trojan defenses [3,4,34,44,46,55], where the attacker-chosen abnormal behavior is to misclassify the input with the trigger as the target label.…”

Section: Trojan Attacks On Dnnsmentioning

confidence: 99%

FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases

Chong¹,

Zhang²,

Ji³

et al. 2023

Preprint

View full text Add to dashboard Cite

Trojan attack on deep neural networks, also known as backdoor attack, is a typical threat to artificial intelligence. A trojaned neural network behaves normally with clean inputs. However, if the input contains a particular trigger, the trojaned model will have attacker-chosen abnormal behavior. Although many backdoor detection methods exist, most of them assume that the defender has access to a set of clean validation samples or samples with the trigger, which may not hold in some crucial real-world cases, e.g., the case where the defender is the maintainer of model-sharing platforms. Thus, in this paper, we propose FREEEAGLE, the first data-free backdoor detection method that can effectively detect complex backdoor attacks on deep neural networks, without relying on the access to any clean samples or samples with the trigger. The evaluation results on diverse datasets and model architectures show that FREEEAGLE is effective against various complex backdoor attacks, even outperforming some state-of-the-art non-data-free backdoor detection methods.

show abstract

“…Most of the current works focus on injecting textual triggers to the context via learning, including characterlevel manipulation [15,37], word-level replacement [15,69], and sentence-level [26,37]. Recent works have been studied towards poisoning language models with adversarial data [6,36,73], inspired by some existing attacks in computer vision [38]. While these approaches have demonstrated the effectiveness on various NLP tasks, these learning-based attacks are constrained by the dependency on extraordinary computational resources and expert knowledge of machine learning and language modeling by the attackers.…”

Section: Backdoor Attacksmentioning

confidence: 99%

Training-free Lexical Backdoor Attacks on Language Models

Huang,

Zhuo,

et al. 2023

Preprint

View full text Add to dashboard Cite

Large-scale language models have achieved tremendous success across various natural language processing (NLP) applications. Nevertheless, language models are vulnerable to backdoor attacks, which inject stealthy triggers into models for steering them to undesirable behaviors. Most existing backdoor attacks, such as data poisoning, require further (re)training or fine-tuning language models to learn the intended backdoor patterns. The additional training process however diminishes the stealthiness of the attacks, as training a language model usually requires long optimization time, a massive amount of data, and considerable modifications to the model parameters.In this work, we propose Training-Free Lexical Backdoor Attack (TFLexAttack) as the first training-free backdoor attack on language models. Our attack is achieved by injecting lexical triggers into the tokenizer of a language model via manipulating its embedding dictionary using carefully designed rules. These rules are explainable to human developers which inspires attacks from a wider range of hackers. The sparse manipulation of the dictionary also habilitates the stealthiness of our attack. We conduct extensive experiments on three dominant NLP tasks based on nine language models to demonstrate the effectiveness and universality of our attack. The code of this work is available at https://github.com/Jinxhy/TFLexAttack. CCS CONCEPTS• Security and privacy → Web application security; • Social and professional topics → social impact; • Computing methodologies → Natural language processing.

show abstract

Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures

Cited by 62 publications

References 46 publications

Aggregation algorithm based on consensus verification

Aggregation algorithm based on consensus verification

FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases

Training-free Lexical Backdoor Attacks on Language Models

Contact Info

Product

Resources

About