User-Level Label Leakage from Gradients in Federated Learning

Abstract

Federated learning enables multiple users to build a joint model by sharing their model updates (gradients), while their raw data remains local on their devices. In contrast to the common belief that this provides privacy benefits, we here add to the very recent results on privacy risks when sharing gradients. Specifically, we investigate Label Leakage from Gradients (LLG), a novel attack to extract the labels of the users’ training data from their shared gradients. The attack exploits the direction and magnitude of gradients to determine the presence or absence of any label. LLG is simple yet effective, capable of leaking potential sensitive information represented by labels, and scales well to arbitrary batch sizes and multiple classes. We mathematically and empirically demonstrate the validity of the attack under different settings. Moreover, empirical results show that LLG successfully extracts labels with high accuracy at the early stages of model training. We also discuss different defense mechanisms against such leakage. Our findings suggest that gradient compression is a practical technique to mitigate the attack.

Federated Learning Attacks Revisited: A Critical Discussion of Gaps, Assumptions, and Evaluation Setups

“…Several model inversion attacks reconstruct the training data by exploiting the shared gradients [ 22 , 78 , 97 ]. In particular, they exploit the mathematical properties of gradients in specific model architectures to infer information about the input data.…”

Privacy-Preserving Federated Recurrent Neural Networks

“…Similarly to tabular or image data, FL was applied on many time-series applications such as load forecasting [36], natural language processing [24], [49], traffic flow prediction, [78], and healthcare systems [42]. However, recent research has shown that sharing gradients and local model updates in FL lead to severe privacy leakage because membership inference and data/label reconstruction attacks are effective [53], [126], [139], [85], [95], [136], [44], [61], [125], [32], [34].…”

“…Although FL provides collaborative learning without data sharing, it still raises privacy issues: The shared intermediate model is vulnerable to privacy attacks that can reconstruct parties' input data or infer the membership of data samples in the training set, hence needs to be protected during the training process [53], [126], [139], [85], [95], [136], [44], [61], [125], [32], [34]. Moreover, it was recently shown that RNNs are particularly vulnerable to inference attacks (e.g., membership inference) compared to traditional neural networks [130].…”

MpFedcon : Model-Contrastive Personalized Federated Learning with the Class Center

“…The iDLG [23] further demonstrates that the last layer of shared gradients must leak ground truth labels when the activation function is non-negative. Wainakh et al [37] further explored the properties of gradient-based leakage of true labels under large batch. Common techniques for protecting privacy include adding noise, gradient compression, discretization, and differential privacypreserving.…”