FedDANE: A Federated Newton-Type Method

Li, Tian; Sahu, Anit Kumar; Zaheer, Manzil; Sanjabi, Maziar; Talwalkar, Ameet; Smithy, Virginia

doi:10.1109/ieeeconf44664.2019.9049023

Cited by 367 publications

(624 citation statements)

References 8 publications

Supporting

Mentioning

615

Contrasting

Unclassified

Order By: Relevance

“…Users may not participate sufficiently in the FL process for several reasons, such as low battery power, poor connection, and so on. The low participation issue during FL model training has been highlighted in several studies 2,68 …”

Section: Discussionmentioning

confidence: 99%

A comprehensive review of federated learning for COVID‐19 detection

Naz

Phan

Chen

2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

The coronavirus of 2019 (COVID‐19) was declared a global pandemic by World Health Organization in March 2020. Effective testing is crucial to slow the spread of the pandemic. Artificial intelligence and machine learning techniques can help COVID‐19 detection using various clinical symptom data. While deep learning (DL) approach requiring centralized data is susceptible to a high risk of data privacy breaches, federated learning (FL) approach resting on decentralized data can preserve data privacy, a critical factor in the health domain. This paper reviews recent advances in applying DL and FL techniques for COVID‐19 detection with a focus on the latter. A model FL implementation use case in health systems with a COVID‐19 detection using chest X‐ray image data sets is studied. We have also reviewed applications of previously published FL experiments for COVID‐19 research to demonstrate the applicability of FL in tackling health research issues. Last, several challenges in FL implementation in the healthcare domain are discussed in terms of potential future work.

show abstract

Section: Discussionmentioning

confidence: 99%

A comprehensive review of federated learning for COVID‐19 detection

Naz

Phan

Chen

2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

show abstract

“…Some of them originate from DANE [12], which is a classical optimization method that introduces a sequence of local subproblems to reduce client drift. Fed-DANE [13] is adapted from DANE by allowing partial device participation. Network-DANE [14] is developed for decentralized federated learning.…”

Section: B Training Variancementioning

confidence: 99%

Differential Privacy Meets Federated Learning Under Communication Constraints

Mohammadi

Bai

Fan

et al. 2022

IEEE Internet Things J.

View full text Add to dashboard Cite

The performance of federated learning systems is bottlenecked by communication costs and training variance. The communication overhead problem is usually addressed by three communication-reduction techniques, namely, model compression, partial device participation, and periodic aggregation, at the cost of increased training variance. Different from traditional distributed learning systems, federated learning suffers from data heterogeneity (since the devices sample their data from possibly different distributions), which induces additional variance among devices during training. Various variance-reduced training algorithms have been introduced to combat the effects of data heterogeneity, while they usually cost additional communication resources to deliver necessary control information. Additionally, data privacy remains a critical issue in FL, and thus there have been attempts at bringing Differential Privacy to this framework as a mediator between utility and privacy requirements. This paper investigates the trade-offs between communication costs and training variance under a resource-constrained federated system theoretically and experimentally, and studies how communication reduction techniques interplay in a differentially private setting. The results provide important insights into designing practical privacy-aware federated learning systems.

show abstract

“…There is another line of work which reduce inter-client variance to eliminate inconsistent update across clients. several approaches use control variates [10,11,18,20] or global gradient momentum [30] to reduce biases in client update. [5,13] apply STORM algorithm [4] to reduce variance caused by both server-level and client-level SGD procedures.…”

Section: Related Workmentioning

confidence: 99%

“…This feature allows the proposed method to achieve the same level of task-specific performance with fewer communication rounds. Moreover, while most of existing methods require additional requirements compared to FedAvg such as full participation [13,20,34], additional communication bandwidth [5,10,11,18,30,36], or storage costs on clients to store local states [1,11,16], FedAGM is completely free from any additional communication and memory overhead, which ensures the compatibility with large-scale and low-participation federated learning scenarios. The main contributions of this paper are summarized as follows.…”

Section: Introductionmentioning

confidence: 99%

Communication-Efficient Federated Learning with Acceleration of Global Momentum

Kim¹,

Kim²,

Han³

2022

Preprint

View full text Add to dashboard Cite

Federated learning often suffers from unstable and slow convergence due to heterogeneous characteristics of participating clients. Such tendency is aggravated when the client participation ratio is low since the information collected from the clients at each round is prone to be more inconsistent. To tackle the challenge, we propose a novel federated learning framework, which improves the stability of the server-side aggregation step, which is achieved by sending the clients an accelerated model estimated with the global gradient to guide the local gradient updates. Our algorithm naturally aggregates and conveys the global update information to participants with no additional communication cost and does not require to store the past models in the clients. We also regularize local update to further reduce the bias and improve the stability of local updates. We perform comprehensive empirical studies on real data under various settings and demonstrate the remarkable performance of the proposed method in terms of accuracy and communication-efficiency compared to the stateof-the-art methods, especially with low client participation rates. Our code is available at https://github.com/ ninigapa0/FedAGM

show abstract

FedDANE: A Federated Newton-Type Method

Cited by 367 publications

References 8 publications

A comprehensive review of federated learning for COVID‐19 detection

A comprehensive review of federated learning for COVID‐19 detection

Differential Privacy Meets Federated Learning Under Communication Constraints

Communication-Efficient Federated Learning with Acceleration of Global Momentum

Contact Info

Product

Resources

About