Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent

Kassab, Rahif; Simeone, Osvaldo

doi:10.48550/arxiv.2009.06419

Cited by 7 publications

(12 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In variational Bayesian federated learning, the agents collectively aim at obtaining a variational distribution q(θ) on the model parameter space that minimizes the global free energy (see, e.g., [11,13])…”

Section: Setupmentioning

confidence: 99%

“…In this section, we review non-parametric variational federated learning as introduced in [13]. Specifically, reference [13] presents DSVGD, which represents the variational posterior q(θ|D) via N particles, {θ n } N n=1 , and extends SVGD [16] to a federated learning setup within the framework of partitioned variational inference (PVI) [11].…”

Section: Particle-based Federated Learningmentioning

confidence: 99%

“…In DSVGD, the server maintains a set of N particles {θ n } N n=1 that are iteratively updated by a subset of agents. We focus here on the case of a single agent scheduled at each iteration, since the extension to more than one agent is direct as detailed in [13]. DSVGD minimizes the global free energy (2) over the variational distribution q(θ) in a distributed manner.…”

Section: Distributed Svgdmentioning

confidence: 99%

“…As such, it is particularly well suited for federated learning applications, in which individual agents have limited data, causing significant levels of epistemic uncertainty, which needs to be quantified in order to ensure reliability and trustworthiness [10]. Federated Bayesian learning protocols have been proposed that apply variational parametric [11,12] and non-parametric [13] Bayesian schemes. Bayesian unlearning was studied in [4,14,15] for centralized settings by introducing variational unlearning criteria.…”

Section: Introductionmentioning

confidence: 99%

“…The novel Bayesian federated unlearning method, referred to as Forget-Stein Variational Gradient Descent (Forget-SVGD). Forget-SVGD builds on SVGD [16] -a particle-based approximate Bayesian inference scheme using gradient-based deterministic updates -and on its distributed (federated) extension known as Distributed SVGD (DSVGD) [13]. Upon the completion of federated learning, as one or more participating agents request for their data to be "forgotten", Forget-SVGD carries out local SVGD updates at the agents whose data need to be "unlearned" interleaved with communication rounds with a parameter server.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Forget-SVGD: Particle-Based Bayesian Federated Unlearning

Gong¹,

Simeone²,

Kassab³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Variational particle-based Bayesian learning methods have the advantage of not being limited by the bias affecting more conventional parametric techniques. This paper proposes to leverage the flexibility of non-parametric Bayesian approximate inference to develop a novel Bayesian federated unlearning method, referred to as Forget-Stein Variational Gradient Descent (Forget-SVGD). Forget-SVGD builds on SVGD -a particle-based approximate Bayesian inference scheme using gradient-based deterministic updates -and on its distributed (federated) extension known as Distributed SVGD (DSVGD). Upon the completion of federated learning, as one or more participating agents request for their data to be "forgotten", Forget-SVGD carries out local SVGD updates at the agents whose data need to be "unlearned", which are interleaved with communication rounds with a parameter server. The proposed method is validated via performance comparisons with non-parametric schemes that train from scratch by excluding data to be forgotten, as well as with existing parametric Bayesian unlearning methods.

show abstract

Section: Setupmentioning

confidence: 99%

Section: Particle-based Federated Learningmentioning

confidence: 99%

Section: Distributed Svgdmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Forget-SVGD: Particle-Based Bayesian Federated Unlearning

Gong¹,

Simeone²,

Kassab³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

A Review of Medical Federated Learning: Applications in Oncology and Cancer Research

Chowdhury

Kassem

Padoy

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Machine learning has revolutionized every facet of human life, while also becoming more accessible and ubiquitous. Its prevalence has had a powerful impact in healthcare, with numerous applications and intelligent systems achieving clinical level expertise. However, building robust and generalizable systems relies on training algorithms in a centralized fashion using large, heterogeneous datasets. In medicine, these datasets are time consuming to annotate and difficult to collect centrally due to privacy concerns. Recently, Federated Learning has been proposed as a distributed learning technique to alleviate many of these privacy concerns by providing a decentralized training paradigm for models using large, distributed data. This new approach has become the defacto way of building machine learning models in multiple industries (e.g. edge computing, smartphones). Due to its strong potential, Federated Learning is also becoming a popular training method in healthcare, where patient privacy is of paramount concern. In this paper we performed an extensive literature review to identify state-of-the-art Federated Learning applications for cancer research and clinical oncology analysis. Our objective is to provide readers with an overview of the evolving Federated Learning landscape, with a focus on applications and algorithms in oncology space. Moreover, we hope that this review will help readers to identify potential needs and future directions for research and development.

show abstract

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Chen

Gündüz

Huang

et al. 2021

IEEE J. Select. Areas Commun.

349

View full text Add to dashboard Cite

The next-generation of wireless networks will enable many machine learning (ML) tools and applications to efficiently analyze various types of data collected by edge devices for inference, autonomy, and decision making purposes. However, due to resource constraints, delay limitations, and privacy challenges, edge devices cannot offload their entire collected datasets to a cloud server for centrally training their ML models or inference purposes. To overcome these challenges, distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges, thus reducing the communication overhead and latency as well as improving data privacy. However, deploying distributed learning over wireless networks faces several challenges including the uncertain wireless environment (e.g., dynamic channel and interference), limited wireless resources (e.g., transmit power and radio spectrum), and hardware resources (e.g., computational power). This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks. We present a detailed overview of several emerging distributed learning paradigms, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning. For each learning framework, we first introduce the motivation for deploying it over wireless networks. Then, we present a detailed literature review on the use of communication techniques for its efficient deployment. We then introduce an illustrative example to show how to optimize wireless networks to improve its performance. Finally, we introduce future research opportunities. In a nutshell, this paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks.

show abstract

Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent

Cited by 7 publications

References 16 publications

Forget-SVGD: Particle-Based Bayesian Federated Unlearning

Forget-SVGD: Particle-Based Bayesian Federated Unlearning

A Review of Medical Federated Learning: Applications in Oncology and Cancer Research

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Contact Info

Product

Resources

About