Backdoor Pre-trained Models Can Transfer to All

Abstract

Pre-trained general-purpose language models have been a dominating component in enabling real-world natural language processing (NLP) applications. However, a pre-trained model with backdoor can be a severe threat to the applications. Most existing backdoor attacks in NLP are conducted in the fine-tuning phase by introducing malicious triggers in the targeted class, thus relying greatly on the prior knowledge of the fine-tuning task. In this paper, we propose a new approach to map the inputs containing triggers directly to a predefined output representation of the pre-trained NLP models, e.g., a predefined output representation for the classification token in BERT, instead of a target label. It can thus introduce backdoor to a wide range of downstream tasks without any prior knowledge. Additionally, in light of the unique properties of triggers in NLP, we propose two new metrics to measure the performance of backdoor attacks in terms of both effectiveness and stealthiness. Our experiments with various types of triggers show that our method is widely applicable to different fine-tuning tasks (classification and named entity recognition) and to different models (such as BERT, XLNet, BART), which poses a severe threat. Furthermore, by collaborating with the popular online model repository Hugging Face, the threat brought by our method has been confirmed. Finally, we analyze the factors that may affect the attack performance and share insights on the causes of the success of our backdoor attack. CCS CONCEPTS• Computing methodologies → Natural language processing; Transfer learning.

A Roadmap for Big Model

“…Backdoor attacks have also been successfully applied to BMs on NLP tasks. For example, [1035] and [1036] simultaneously propose backdoor attacks on pre-trained NLP models, such that the downstream tasks after fine-tuning can also inherent the backdoor behavior. [1037] propose a weight poisoning approach that the pre-trained weights are injected with vulnerabilities which expose backdoors after fine-tuning.…”

“…Recent studies demonstrated adding a few specially-crafted data to the training corpus can manipulate the model, e.g., generating offensive text [1092,1016], wrong translations [1093] or suggesting insecure code [1090]. Moreover, the backdoors in the pretrained language models can impact a wide range of downstream tasks [1035]. It makes the pretrained model a single point of failure for all downstream applications.…”

A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks

“…Triggers are attacker-specific patterns that activate backdoors. Most backdoor triggers are fixed words [11,15,33,38,44] or sentences [6]. To make triggers invisible, some attackers design syntactic [26] or style [25] triggers, where backdoors activate when input texts of certain syntax or style.…”

“…Users get a model specifically trained for the task and further tune it on clean datasets. Finally, with the weak assumption that no task-specific knowledge is available, some attackers use plain texts to attack general-purpose PLMs and leave the backdoor to arbitrary downstream tasks [44,33].…”

Transfer Attacks Revisited: A Large-Scale Empirical Study in Real Computer Vision Settings

“…Therefore, understanding the deciding factors of transferability and their working mechanisms has attracted intensive researches [15], [19], [29], [34], [36], [39], [44]. However, to the best of our knowledge, all systematic empirical studies are conducted under controlled "lab" environments that are too ideal to make the derived conclusions reliable in the real environment.…”