Improved Gradient Inversion Attacks and Defenses in Federated Learning

Geng, Jiahui; Mou, Yongli; Li, Qing; Li, Feifei; Beyan, Oya; Decker, Stefan; Rong, Chunming

doi:10.1109/tbdata.2023.3239116

Cited by 28 publications

(22 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Te virtual data are learned using an optimization algorithm in such a way that the gradient obtained by backpropagation on the common model is similar to the real gradient, and the training data and labels are obtained after several rounds of iterative optimization. At the moment, this is one of the hottest topics in the study of variants of DLG-based methods [9,[52][53][54].…”

Section: Gradient Update-based Data Leakagementioning

confidence: 99%

“…FL builds iterable aggregated models by training distributed models across multiple data sources with local data, only exchanging model parameters or intermediate results models, and thus learning a shared target model. Improved approaches based on FL methods [2][3][4][5][6][7][8][9] have carried out a lot of research work in achieving a balance between data privacy protection and data sharing computation. Currently, more researchers are using cryptographic privacy-preserving methods and diferential privacy-preserving methods to achieve privacy-preserving security for local gradients in the federal learning security problem [10,11].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fast and Accurate Deep Leakage from Gradients Based on Wasserstein Distance

Peng

Tan

2023

International Journal of Intelligent Systems

View full text Add to dashboard Cite

Shared gradients are widely used to protect the private information of training data in distributed machine learning systems. However, Deep Leakage from Gradients (DLG) research has found that private training data can be recovered from shared gradients. The DLG method still has some issues such as the “Exploding Gradient,” low attack success rate, and low fidelity of recovered data. In this study, a Wasserstein DLG method, named WDLG, is proposed; the theoretical analysis shows that under the premise that the output layer of the model has a “bias” term, predicting the “label” of the data by whether the “bias” is “negative” or not is independent of the approximation of the shared gradient, and thus, the label of the data can be recovered with 100% accuracy. In the proposed method, the Wasserstein distance is used to calculate the error loss between the shared gradient and the virtual gradient, which improves model training stability, solves the “Exploding Gradient” phenomenon, and improves the fidelity of the recovered data. Moreover, a large learning rate strategy is designed to improve model training convergence speed in-depth. Finally, the WDLG method is validated on datasets from MNIST, Fashion MNIST, SVHN, CIFAR-100, and LFW. Experiments results show that the proposed WDLG method provides more stable updates for virtual data, a higher attack success rate, faster model convergence, higher image fidelity during recovery, and support for designing large learning rate strategies.

show abstract

Section: Gradient Update-based Data Leakagementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fast and Accurate Deep Leakage from Gradients Based on Wasserstein Distance

Peng

Tan

2023

International Journal of Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…We choose the result with the best performance of each method in comparison. The batch size is set to 128, which is considered privacy-preserving enough in Federated Learning [21], since using a small batch size would elevate the risk of gradient inverting attacks from malicious clients or an unreliable server.…”

Section: Tackling the Client Dropoutmentioning

confidence: 99%

PerFedNILM: A Practical Personalized Federated Learning-based non-Intrusive Load Monitoring

Pan,

Wang,

et al. 2023

Preprint

View full text Add to dashboard Cite

Non-Intrusive Load Monitoring (NILM) is a valuable technique for breaking down overall power consumption into the energy usage of individual appliances. Understanding power usage patterns through NILM plays an important role in reducing energy costs and achieving carbon reduction goals. However, privacy concerns often deter consumers from sharing their electricity consumption data. To address these privacy concerns, Federated Learning (FL) has been introduced in NILM, which enables the training of NILM models while keeping power consumers' data locally. However, FL's reliance on a single global model leads to poor performance on clients with unique power consumption patterns. In response to this challenge, we present a Personalized Federated Learning NILM algorithm (PerFedNILM), a practical personalized FL approach for NILM. PerFedNILM limits the local update bias across clients and trains personalized models for individual clients to improve load monitoring performance. Additionally, it mitigates the negative impact of client dropout, which is a common issue in practice. Our experiments on using real-world energy data demonstrate that PerFedNILM outperforms previous FL-based NILM methods, especially in client dropout scenarios.

show abstract

“…In the application of MR reconstruction using FL, Guo et al [24] tried to address the domain shift issue by iteratively aligning the latent feature of UNet [25] between target and other client sites. However, their cross-site strategy requires the target client to share both the latent feature and the network parameter with other client sites in each communication rounds, which could result in data privacy concerns [26], [27]. Moreover, the cross-site strategy requires communications between local clients with their local data, which may contradict the purpose of FL.…”

Section: Introductionmentioning

confidence: 99%

Federated Transfer Learning for Low-Dose PET Denoising: A Pilot Study With Simulated Heterogeneous Data

Zhou

Miao

Mirian

et al. 2023

IEEE Trans. Radiat. Plasma Med. Sci.

View full text Add to dashboard Cite

Positron emission tomography (PET) with a reduced injection dose, i.e., low-dose PET, is an efficient way to reduce radiation dose. However, low-dose PET reconstruction suffers from a low signal-to-noise ratio (SNR), affecting diagnosis and other PET-related applications. Recently, deep learning-based PET denoising methods have demonstrated superior performance in generating high-quality reconstruction. However, these methods require a large amount of representative data for training, which can be difficult to collect and share due to medical data privacy regulations. Moreover, low-dose PET data at different institutions may use different low-dose protocols, leading to nonidentical data distribution. While previous federated learning (FL) algorithms enable multiinstitution collaborative training without the need of aggregating local data, it is challenging for previous methods to address the large domain shift caused by different low-dose PET settings, and the application of FL to PET is still underexplored. In this work, we propose a federated transfer learning (FTL) framework for low-dose PET denoising using heterogeneous low-dose data. Our experimental results on simulated multiinstitutional data demonstrate that our method can efficiently utilize heterogeneous low-dose data without compromising data privacy for achieving superior low-dose PET

show abstract

Improved Gradient Inversion Attacks and Defenses in Federated Learning

Cited by 28 publications

References 20 publications

Fast and Accurate Deep Leakage from Gradients Based on Wasserstein Distance

Fast and Accurate Deep Leakage from Gradients Based on Wasserstein Distance

PerFedNILM: A Practical Personalized Federated Learning-based non-Intrusive Load Monitoring

Federated Transfer Learning for Low-Dose PET Denoising: A Pilot Study With Simulated Heterogeneous Data

Contact Info

Product

Resources

About