Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning

Abstract

Federated learning, i.e., a mobile edge computing framework for deep learning, is a recent advance in privacypreserving machine learning, where the model is trained in a decentralized manner by the clients, i.e., data curators, preventing the server from directly accessing those private data from the clients. This learning mechanism significantly challenges the attack from the server side. Although the state-ofthe-art attacking techniques that incorporated the advance of Generative adversarial networks (GANs) could construct class representatives of the global data distribution among all clients, it is still challenging to distinguishably attack a specific client (i.e., user-level privacy leakage), which is a stronger privacy threat to precisely recover the private data from a specific client. This paper gives the first attempt to explore user-level privacy leakage against the federated learning by the attack from a malicious server. We propose a framework incorporating GAN with a multitask discriminator, which simultaneously discriminates category, reality, and client identity of input samples. The novel discrimination on client identity enables the generator to recover user specified private data. Unlike existing works that tend to interfere the training process of the federated learning, the proposed method works "invisibly" on the server side. The experimental results demonstrate the effectiveness of the proposed attacking approach and the superior to the state-of-the-art. 1

A Crowdsourcing Framework for On-Device Federated Learning

“…We observe that Lemma 2 is a direct consequence of the solution structure derived in (28). Hence, we conclude the proof.…”

“…The idea in this work was to adapt the communication period such that it minimizes the optimization error at each wall-clock time. To this end, interestingly, in some of the latest works such as [28], the authors have well-studied and demonstrated the privacy risk scenario under collaborated learning mechanism such as FL.…”

The future of digital health with federated learning

“…Information leakage: By definition, FL systems avoid sharing healthcare data among participating institutions. However, the shared information may still indirectly expose private data used for local training, e.g., by model inversion 60 of the model updates, the gradients themselves 61 or adversarial attacks 62,63 . FL is different from traditional training insofar as the training process is exposed to multiple parties, thereby increasing the risk of leakage via reverseengineering if adversaries can observe model changes over time, observe specific model updates (i.e., a single institution's update), or manipulate the model (e.g., induce additional memorisation by others through gradient-ascent-style attacks).…”

Trustworthy artificial intelligence

“…Future research should thus seek to explore the application of DLT-based federated learning to more complex AI models and investigate ways to reduce the induced performance overhead in real-world application scenarios. Furthermore, despite increased confidentiality, research has also shown that federated learning is potentially vulnerable to inference attacks, whereby an adversary can aim to extract information about private training data by inferring the AI model multiple times (Melis et al 2019;Wang et al 2019). In addition to employing DLT for preserving training data provenance and AI model integrity, future research should therefore also explore how DLT could help with preventing inference attacks on federated learning networks.…”