When the Curious Abandon Honesty: Federated Learning Is Not Private

Boenisch, Franziska; Dziedzic, Adam; Schuster, Roei; Shamsabadi, Ali Shahin; Shumailov, Ilia; Papernot, Nicolas

doi:10.48550/arxiv.2112.02918

Cited by 18 publications

(31 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, it provides some level of privacy protection. However, recent works in FL have shown that model parameters exchanged during training could leak the raw data [5]. This remains an open challenge with FLAME as well.…”

Section: Similarity-based Clientmentioning

confidence: 99%

FLAME: Federated Learning Across Multi-device Environments

Cho¹,

Mathur²,

Kawsar³

2022

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) enables distributed training of machine learning models while keeping personal data on user devices private. While we witness increasing applications of FL in the area of mobile sensing, such as human-activity recognition, FL has not been studied in the context of a multi-device environment (MDE), wherein each user owns multiple data-producing devices. With the proliferation of mobile and wearable devices, MDEs are increasingly becoming popular in ubicomp settings, therefore necessitating the study of FL in them. FL in MDEs is characterized by high non-IID-ness across clients, complicated by the presence of both user and device heterogeneities. Further, ensuring efficient utilization of system resources on FL clients in a MDE remains an important challenge. In this paper, we propose FLAME, a user-centered FL training approach to counter statistical and system heterogeneity in MDEs, and bring consistency in inference performance across devices. FLAME features (i) user-centered FL training utilizing the time alignment across devices from the same user; (ii) accuracy-and efficiency-aware device selection; and (iii) model personalization to devices. We also present an FL evaluation testbed with realistic energy drain and network bandwidth profiles, and a novel class-based data partitioning scheme to extend existing HAR datasets to a federated setup. Our experiment results on three multi-device HAR datasets show that FLAME outperforms various baselines by 4.8-33.8% higher 𝐹 1 score, 1.02-2.86× greater energy efficiency, and up to 2.02× speedup in convergence to target accuracy through fair distribution of the FL workload.CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing systems and tools; • Computing methodologies → Cooperation and coordination.

show abstract

Section: Similarity-based Clientmentioning

confidence: 99%

FLAME: Federated Learning Across Multi-device Environments

Cho¹,

Mathur²,

Kawsar³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, this previous work often requires unrealistic model modifications. Attacks in Fowl et al (2021) and Boenisch et al (2021) can only recover data from linear layers and thus require the existence of large linear layers -even one of which could be larger than standard vision models -early in a network, to avoid downsampling and striding operations that degrade information. The attack in Pasquini et al (2021) requires the ability of the server to send separate malicious parameters to individual users who each own only few data points -a threat that can in turn be overcome quickly if aggregation protocols are used in reverse to average server updates before processing them on the user side.…”

Section: Background and Related Workmentioning

confidence: 99%

“…The attacks in Fowl et al (2021); Boenisch et al (2021) and Pasquini et al (2021) can collectively be understood as attacks based on gradient sparsity. Although the secure aggregation protocol nominally runs as it was designed, all but one of the data points (Boenisch et al, 2021) or users (Pasquini et al, 2021) return a zero gradient for some parameters in the model, such that averaging still returns these entries directly. In Fowl et al (2021), individual gradients are non-zero, but parts of the gradient obey a cumulative sum structure from which a sparse single update can be recovered.…”

Section: Background and Related Workmentioning

confidence: 99%

“…However, other forms of protection, or at least identification, may be possible. The attack investigated in this work, and the attacks of Boenisch et al (2021) and Pasquini et al (2021) work by producing sparse gradients. This means that the update returned by the user contains parameters where their gradients are dominated (or entirely contributed) by single examples.…”

Section: Defenses and Mitigation Strategiesmentioning

confidence: 99%

“…Previously, such attacks have been demonstrated mostly on either small batches of data, e.g. in Wang et al (2018); Zhu et al (2019) under the threat model of an "honest-but-curious" server or in settings with "malicious model" threat models with modified architectures (Fowl et al, 2021;Boenisch et al, 2021). For the threat model of an unmodified architecture, existing work seems to suggest that even a moderate batch size of 32 could be sufficient to deter an attack (Huang et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fishing for User Data in Large-Batch Federated Learning via Gradient Magnification

Wen¹,

Geiping²,

Fowl³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) has rapidly risen in popularity due to its promise of privacy and efficiency. Previous works have exposed privacy vulnerabilities in the FL pipeline by recovering user data from gradient updates. However, existing attacks fail to address realistic settings because they either 1) require a 'toy' settings with very small batch sizes, or 2) require unrealistic and conspicuous architecture modifications. We introduce a new strategy that dramatically elevates existing attacks to operate on batches of arbitrarily large size, and without architectural modifications. Our model-agnostic strategy only requires modifications to the model parameters sent to the user, which is a realistic threat model in many scenarios. We demonstrate the strategy in challenging large-scale settings, obtaining high-fidelity data extraction in both cross-device and cross-silo federated learning. * Authors contributed equally. Order chosen randomly.

show abstract

Mimicking clinical trials with synthetic acute myeloid leukemia patients using generative artificial intelligence

Eckardt,

Hahn,

Röllig

et al. 2024

npj Digit. Med.

View full text Add to dashboard Cite

Clinical research relies on high-quality patient data, however, obtaining big data sets is costly and access to existing data is often hindered by privacy and regulatory concerns. Synthetic data generation holds the promise of effectively bypassing these boundaries allowing for simplified data accessibility and the prospect of synthetic control cohorts. We employed two different methodologies of generative artificial intelligence – CTAB-GAN+ and normalizing flows (NFlow) – to synthesize patient data derived from 1606 patients with acute myeloid leukemia, a heterogeneous hematological malignancy, that were treated within four multicenter clinical trials. Both generative models accurately captured distributions of demographic, laboratory, molecular and cytogenetic variables, as well as patient outcomes yielding high performance scores regarding fidelity and usability of both synthetic cohorts (n = 1606 each). Survival analysis demonstrated close resemblance of survival curves between original and synthetic cohorts. Inter-variable relationships were preserved in univariable outcome analysis enabling explorative analysis in our synthetic data. Additionally, training sample privacy is safeguarded mitigating possible patient re-identification, which we quantified using Hamming distances. We provide not only a proof-of-concept for synthetic data generation in multimodal clinical data for rare diseases, but also full public access to synthetic data sets to foster further research.

show abstract

When the Curious Abandon Honesty: Federated Learning Is Not Private

Cited by 18 publications

References 0 publications

FLAME: Federated Learning Across Multi-device Environments

FLAME: Federated Learning Across Multi-device Environments

Fishing for User Data in Large-Batch Federated Learning via Gradient Magnification

Mimicking clinical trials with synthetic acute myeloid leukemia patients using generative artificial intelligence

Contact Info

Product

Resources

About