Differential Privacy Meets Federated Learning Under Communication Constraints

Mohammadi, Nima; Bai, Jianan; Fan, Qi; Song, Yifei; Yi, Yang; Liu, Lingjia

doi:10.1109/jiot.2021.3101991

Cited by 26 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the assurance of local user data privacy is a standout feature of FL, malicious actors may still be able to glean critical system information from model changes [10]. Although newer methods such as secure multiparty computation (SMC) [11], differential privacy [12] or secure aggregation [5] seek to improve the privacy of FL, these approaches generally sacrifice inference performance for privacy. Understanding and balancing these costs is a significant difficulty in implementing private FL systems, both theoretically and practically [13].…”

Section: B Privacy Issues In Flmentioning

confidence: 99%

Distributed Learning Meets 6G: A Communication and Computing Perspective

Jere¹,

Song²,

Yang³

et al. 2023

Preprint

View full text Add to dashboard Cite

With the ever-improving computing capabilities and storage capacities of mobile devices in line with evolving telecommunication network paradigms, there has been an explosion of research interest towards exploring Distributed Learning (DL) frameworks to realize stringent key performance indicators (KPIs) that are expected in next-generation/6G cellular networks. In conjunction with Edge Computing, Federated Learning (FL) has emerged as the DL architecture of choice in prominent wireless applications. This article lays an outline of how DL in general and FL-based strategies specifically can contribute towards realizing a part of the 6G vision and strike a balance between communication and computing constraints. As a practical use case, we apply Multi-Agent Reinforcement Learning (MARL) within the FL framework to the Dynamic Spectrum Access (DSA) problem and present preliminary evaluation results. Top contemporary challenges in applying DL approaches to 6G networks are also highlighted.

show abstract

Section: B Privacy Issues In Flmentioning

confidence: 99%

Distributed Learning Meets 6G: A Communication and Computing Perspective

Jere¹,

Song²,

Yang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition to that, most of the privacy-preserving distributed learning methods mentioned above consider periodical local model updates from the local devices and then send the aggregated model back to all devices. This results in substantial communication overhead [25]. While considering both aspects, we propose a highperforming ensemble distributed learning system with knowledge transfer based on differential privacy, which does not require a continuous local model update.…”

Section: Related Workmentioning

confidence: 99%

Privacy-Preserving Ensemble Infused Enhanced Deep Neural Network Framework for Edge Cloud Convergence

Stephanie

Khalil

Rahman

et al. 2023

IEEE Internet Things J.

View full text Add to dashboard Cite

We propose a privacy-preserving ensemble infused enhanced Deep Neural Network (DNN) based learning framework in this paper for Internet-of-Things (IoT), edge, and cloud convergence in the context of healthcare. In the convergence, edge server is used for both storing IoT produced bioimage and hosting DNN algorithm for local model training. The cloud is used for ensembling local models. The DNN-based training process of a model with a local dataset suffers from low accuracy, which can be improved by the aforementioned convergence and Ensemble Learning. The ensemble learning allows multiple participants to outsource their local model for producing a generalized final model with high accuracy. Nevertheless, Ensemble Learning elevates the risk of leaking sensitive private data from the final model. The proposed framework presents a Differential Privacybased privacy-preserving DNN with Transfer Learning for a local model generation to ensure minimal loss and higher efficiency at edge server. We conduct several experiments to evaluate the performance of our proposed framework.

show abstract

“…Concretely, non-iid data requires more communication rounds to achieve the desired performance but more rounds mean more noise addition which degrades the utility. To our best knowledge, only a few papers [12]- [14] simultaneously investigate the non-iid and DP problem. These works either utilize the full client sampling or MD client sampling that ignores the impact of the heterogeneous client due to non-iid data.…”

Section: Introductionmentioning

confidence: 99%

Differentially Private Federated Learning on Non-iid Data: Convergence Analysis and Adaptive Optimization

Chen,

Ding,

Bao

et al. 2024

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Federated learning (FL) has attracted increasing attention in recent years due to its data privacy preservation and great applicability to large-scale user scenarios. However, when FL faces numerous clients, it is inevitable to emerge the non-independent and identically distributed (non-iid) data between clients, which brings an enormous challenge for model training and performance analysis like convergence. Besides, due to the non-iid data, the participating clients of FL tend to be extremely heterogeneous so the number of samplings among clients causes a sampling variance problem, which induces a huge variation in convergence. More importantly, although FL can foster privacy security via locally retaining the training data, if local data is secret and sensitive, FL should have more powerful privacy protection to resist the cloud server or third party to infer private information from shared models or intermediate gradients. Facing the non-iid and privacy challenges, we propose a differential privacy (DP) based non-iid FL algorithm called DPNFL to jointly tackle these two issues. Specifically, motivated by the DP and its variants, we are the first to adopt the truncated concentrated differential privacy technique under the FL scenario to more tightly track end-to-end privacy loss, while requiring less noise injection for the same level of DP. To avoid the sampling variance problem, we enable the server to sample the partial clients uniformly without replacement, which also guarantees unbiased sampling. To further improve the algorithm performance, we also propose an adaptive version of DPNFL named AdDPNFL, which adopts the adaptive optimization on the server-side to simultaneously alleviate the impact of non-iid data and DP noise on model utility. Finally, we perform extensive experiments to validate the effectiveness and superiority of our algorithms.

show abstract

Differential Privacy Meets Federated Learning Under Communication Constraints

Cited by 26 publications

References 18 publications

Distributed Learning Meets 6G: A Communication and Computing Perspective

Distributed Learning Meets 6G: A Communication and Computing Perspective

Privacy-Preserving Ensemble Infused Enhanced Deep Neural Network Framework for Edge Cloud Convergence

Differentially Private Federated Learning on Non-iid Data: Convergence Analysis and Adaptive Optimization

Contact Info

Product

Resources

About