Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!

Abstract

Natural language processing (NLP) tasks, ranging from text classification to text generation, have been revolutionised by the pretrained language models, such as BERT. This allows corporations to easily build powerful APIs by encapsulating fine-tuned BERT models for downstream tasks. However, when a fine-tuned BERT model is deployed as a service, it may suffer from different attacks launched by the malicious users. In this work, we first present how an adversary can steal a BERT-based API service (the victim/target model) on multiple benchmark datasets with limited prior knowledge and queries. We further show that the extracted model can lead to highly transferable adversarial attacks against the victim model. Our studies indicate that the potential vulnerabilities of BERT-based API services still hold, even when there is an architectural mismatch between the victim model and the attack model. Finally, we investigate two defence strategies to protect the victim model, and find that unless the performance of the victim model is sacrificed, both model extraction and adversarial transferability can effectively compromise the target models.

Protecting Intellectual Property of Language Generation APIs with Lexical Watermark

“…Model extraction attack (MEA) or imitation attack has received significant attention in the past years (Tramèr et al 2016;Correia-Silva et al 2018;Wallace, Stern, and Song 2020;Krishna et al 2020;He et al 2021a;Xu et al 2021). MEA aims to imitate the functionality of a black-box victim model.…”

“…V can process customer queries and return the predictions y as its response. Note that y is a predicted label or a probability vector, if T is a classification problem (Krishna et al 2020;Szyller et al 2021;He et al 2021a). If T is a generation task, y can be a sequence of tokens (Wallace, Stern, and Song 2020;Xu et al 2021).…”

“…As a byproduct of the Machine-learning-as-a-service (MLaaS) paradigm, it is believed that companies could prevent customers from redistributing models to illegitimate users. Nevertheless, a series of emerging model extraction attacks have validated that the functionality of the victim API can be stolen with carefully-designed queries, causing IP infringement (Tramèr et al 2016;Wallace, Stern, and Song 2020;Krishna et al 2020;He et al 2021a). Such attacks have been demonstrated to be effective on not only laboratory models, but also commercial APIs (Wallace, Stern, and Song 2020;Xu et al 2021).…”

A Roadmap for Big Model

“…Recent works have focused on discussing the severe robustness problem of BLMs, mainly containing two types: (1) Adversarial Attacking, which generates new samples by small perturbation on the original inputs to mislead the BLMs's into making wrong predictions. Current works utilize the model prediction, prediction probabilities, and model gradients of the fine-tuned BLMs to search adversarial examples, from char-level attacking [460], word-level attacking [461,462,463,464,465] , sentence-level attacking [466,467] to multi-level attacking [468,469], showing that the robustness of BLMs to adversarial attacking is still far from perfect; (2) Backdoor Attacking, which inserts instances with specifically designed patterns into training data so that the trained BLMs may perform well on normal samples but behave badly on those samples with these patterns. Existing backdoor attacking works of big models mainly focus on exploring more types of triggers [470], data-free backdoor attacking [471], effectiveness on clean sets [472], effectiveness after fine-tuning [473,474] and stealth attacking [475].…”

CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks

“…Unfortunately, previous works have validated that the functionality of a victim API can be stolen through imitation attacks, which inquire the victim with carefully designed queries and train an imitation model based on the outputs of the target API. Such attacks cause severe IP violations of the target API and stifle the creativity and motivation of our research community [44,48,20,12]. Figure 1: Ratio change of word frequency of top 100 words between benign and watermarked corpora used by [13], namely P b (w)/P w (w).…”