TAG: Gradient Attack on Transformer-based Language Models

Abstract

Although distributed learning has increasingly gained attention in terms of effectively utilizing local devices for data privacy enhancement, recent studies show that publicly shared gradients in the training process can reveal the private training data (gradient leakage) to a third party. However, so far there hasn't been any systematic study of the gradient leakage mechanism of the Transformer based language models. In this paper, as the first attempt, we formulate the gradient attack problem on the Transformer-based language models and propose a gradient attack algorithm, TAG, to recover the local training data. Experimental results on Transformer, TinyBERT 4 , TinyBERT 6 , BERT BASE , and BERT LARGE using GLUE benchmark show that compared with DLG (Zhu et al., 2019), TAG works well on more weight distributions in recovering private training data and achieves 1.5× Recover Rate and 2.5× ROUGE-2 over prior methods without the need of ground truth label. TAG can obtain up to 88.9% tokens and up to 0.93 cosine similarity in token embeddings from private training data by attacking gradients on CoLA dataset. In addition, TAG is stronger than previous approaches on larger models, smaller dictionary size, and smaller input length.

LAMP: Extracting Text from Gradients with Language Model Priors

“…Despite substantial progress on image reconstruction, attacks in other domains remain challenging, as the techniques used for images rely extensively on domain specific knowledge. In the domain of text, in particular, where federated learning is often applied (Shejwalkar et al, 2021), only a handful of works exist (Zhu et al, 2019;Deng et al, 2021;Lu et al, 2021). DLG (Zhu et al, 2019) was first to attempt reconstruction from gradients coming from a transformer; TAG (Deng et al, 2021) extended DLG by adding an L 1 term to the reconstruction loss; finally, unlike TAG and DLG which are optimization-based techniques, APRIL (Lu et al, 2021) recently demonstrated an exact gradient leakage technique applicable to transformer networks.…”

“…In the domain of text, in particular, where federated learning is often applied (Shejwalkar et al, 2021), only a handful of works exist (Zhu et al, 2019;Deng et al, 2021;Lu et al, 2021). DLG (Zhu et al, 2019) was first to attempt reconstruction from gradients coming from a transformer; TAG (Deng et al, 2021) extended DLG by adding an L 1 term to the reconstruction loss; finally, unlike TAG and DLG which are optimization-based techniques, APRIL (Lu et al, 2021) recently demonstrated an exact gradient leakage technique applicable to transformer networks. However, APRIL assumes batch size of 1 and learnable positional embeddings, which makes it easy to defend against.…”

“…Gradient leakage attacks usually assume honest-but-curious servers which are not allowed to modify the federated training protocol outlined above. A common approach to gradient leakage attacks, adopted by Zhu et al (2019); Zhao et al (2020); Deng et al (2021) as well as our work, is to obtain the private data by solving the optimization problem:…”

“…where δ is some distance measure and (x c , y c ) denotes dummy data optimized using gradient descent to have similar gradients ∇ θ k L(x c , y c ) to the true data (x * c , y * c ). Common choices for d are L 2 (Zhu et al, 2019), L 1 (Deng et al, 2021) and cosine distances (Geiping et al, 2020). When the true label of the data y * c is known, the problem simplifies to…”

“…These priors guide the reconstruction towards natural images more likely to correspond to client data. However, the use of priors has so far been missing from attacks on text (Zhu et al, 2019;Deng et al, 2021), limiting their ability to reconstruct real client data. This Work In this work, we propose LAMP, a new reconstruction attack which leverages language model priors to extract private text from gradients.…”