Private Wireless Federated Learning with Anonymous Over-the-Air Computation

Hasırcıoglu, Burak; Gündüz, Deniz

doi:10.1109/icassp39728.2021.9413624

Cited by 23 publications

(13 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, the authors designed a private wireless gradient aggregation scheme that relies on the device selection scheme to improve differential privacy. While these works benefit mainly from the presence of channel noise, and depend critically on the perfect channel knowledge at the transmitters, in [122], the authors exploit the anonymity provided by AirComp for privacy, which prevents the PS to detect which devices are participating in each round.…”

Section: State-of-the-art and Research Opportunitiesmentioning

confidence: 99%

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Chen

Gündüz

Huang

et al. 2021

IEEE J. Select. Areas Commun.

Self Cite

353

View full text Add to dashboard Cite

The next-generation of wireless networks will enable many machine learning (ML) tools and applications to efficiently analyze various types of data collected by edge devices for inference, autonomy, and decision making purposes. However, due to resource constraints, delay limitations, and privacy challenges, edge devices cannot offload their entire collected datasets to a cloud server for centrally training their ML models or inference purposes. To overcome these challenges, distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges, thus reducing the communication overhead and latency as well as improving data privacy. However, deploying distributed learning over wireless networks faces several challenges including the uncertain wireless environment (e.g., dynamic channel and interference), limited wireless resources (e.g., transmit power and radio spectrum), and hardware resources (e.g., computational power). This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks. We present a detailed overview of several emerging distributed learning paradigms, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning. For each learning framework, we first introduce the motivation for deploying it over wireless networks. Then, we present a detailed literature review on the use of communication techniques for its efficient deployment. We then introduce an illustrative example to show how to optimize wireless networks to improve its performance. Finally, we introduce future research opportunities. In a nutshell, this paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks.

show abstract

Section: State-of-the-art and Research Opportunitiesmentioning

confidence: 99%

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Chen

Gündüz

Huang

et al. 2021

IEEE J. Select. Areas Commun.

Self Cite

353

View full text Add to dashboard Cite

show abstract

“…When the server is untrustworthy, CSI obtained from the server can be tampered to lurk users to leak information. Therefore, [184] and [185] study the case when CSI is not available.…”

Section: B Differential Private Federated Learningmentioning

confidence: 99%

Private Retrieval, Computing and Learning: Recent Progress and Future Challenges

Ulukuş¹,

Avestimehr²,

Gastpar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Most of our lives are conducted in the cyberspace. The human notion of privacy translates into a cyber notion of privacy on many functions that take place in the cyberspace. This article focuses on three such functions: how to privately retrieve information from cyberspace (privacy in information retrieval), how to privately leverage large-scale distributed/parallel processing (privacy in distributed computing), and how to learn/train machine learning models from private data spread across multiple users (privacy in distributed (federated) learning). The article motivates each privacy setting, describes the problem formulation, summarizes breakthrough results in the history of each problem, and gives recent results and discusses some of the major ideas that emerged in each field. In addition, the crosscutting techniques and interconnections between the three topics are discussed along with a set of open problems and challenges. I. INTRODUCTIONPrivacy is an important part of human life. This article considers privacy in the context of three distinct but related engineering applications, namely, privacy in retrieving information, privacy in computing functions, and privacy in learning. In the first sub-topic of private information retrieval, a user wishes to download a content from publicly accessible databases in such a way that the databases do not learn which particular content the user has downloaded. Towards that goal, the user creates ambiguity by its actions during the download. This strategy prevents databases from guessing which content the user has downloaded. This in turn preserves the user's privacy because what is downloaded leaks information about interest and intent on the part of the user. In the second sub-topic of private computing, a user wishes to compute a function but

show abstract

“…Communication quality at the wireless edge as a key design principle is considered in the federated edge learning (FEEL) framework [10], which takes into account the wireless channel characteristics from the clients to the PS to optimize the convergence and final performance of the global model at the PS. So far, the FEEL paradigm has mainly focused on direct communication from the clients to the PS, and aimed at improving the performance by optimizing resource allocation across clients [10][11][12][13][14][15][16][17][18][19][20]; this model ignores possible cooperation amongst clients in the case of intermittent communication blockages.…”

Section: Introductionmentioning

confidence: 99%

“…Within this framework, an original and promising approach is the over-the-air computation (OAC) strategy [15][16][17], which exploits the signal superposition property of the wireless medium to convey the sum of the model updates in an uncoded fashion. In addition to bandwith efficiency, the OAC framework also provides a certain level of anonymity to the clients due to its superposition nature, and hence, enhances the privacy of the participating clients [18,19]. We emphasize here that in the OAC framework, the PS receives the aggregate model, and it is not possible to disentangle the individual model updates.…”

Section: Introductionmentioning

confidence: 99%

Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying

Yemini¹,

Saha²,

Ozfatura³

et al. 2022

Preprint

View full text Add to dashboard Cite

Intermittent client connectivity is one of the major challenges in centralized federated edge learning frameworks. Intermittently failing uplinks to the central parameter server (PS) can induce a large generalization gap in performance especially when the data distribution among the clients exhibits heterogeneity. In this work, to mitigate communication blockages between clients and the central PS, we introduce the concept of knowledge relaying wherein the successfully participating clients collaborate in relaying their neighbors' local updates to a central parameter server (PS) in order to boost the participation of clients with intermittently failing connectivity. We propose a collaborative relaying based semi-decentralized federated edge learning framework where at every communication round each client first computes a local consensus of the updates from its neighboring clients and eventually transmits a weighted average of its own update and those of its neighbors to the PS. We appropriately optimize these averaging weights to reduce the variance of the global update at the PS while ensuring that the global update is unbiased, consequently improving the convergence rate. Finally, by conducting experiments on CIFAR-10 dataset we validate our theoretical results and demonstrate that our proposed scheme is superior to Federated averaging benchmark especially when data distribution among clients is non-iid.* Michal Yemini and Andrea J. Goldsmith are with the Faculty of Electrical and Computer Engineering, Princeton University.

show abstract

Private Wireless Federated Learning with Anonymous Over-the-Air Computation

Cited by 23 publications

References 15 publications

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Private Retrieval, Computing and Learning: Recent Progress and Future Challenges

Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying

Contact Info

Product

Resources

About