SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead

Wu, Wentai; He, Ligang; Lin, Weiwei; Mao, Rui; Maple, Carsten; Jarvis, Stephen A.

doi:10.1109/tc.2020.2994391

Cited by 251 publications

(100 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, few work studies how to identify the non-IID degrees of the data at clients. Although client selection problem [25]- [28] has also been studied in the literature, none of them considers it from the perspective of their non-IID degrees of data. For example, [25] formulates a joint learning, wireless resource allocation, and user selection problem, to minimize the FL loss function.…”

Section: Introductionmentioning

confidence: 99%

“…The impact of the number of clients and communication rounds, on CIFAR-10 dataset.according to their training and communication latency [27]. studies incentive mechanisms for participating in training and client selection, so as to provide reliable federated learning [28]. proposeS SAFA, a semi-asynchronous FL protocol, to address the problems such as low round efficiency and poor convergence rate in extreme weak communication conditions.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

et al. 2021

View full text Add to dashboard Cite

Federated Learning (FL) has recently attracted considerable attention in internet of things, due to its capability of enabling mobile clients to collaboratively learn a global prediction model without sharing their privacy-sensitive data to the server. Despite its great potential, a main challenge of FL is that the training data are usually non-Independent, Identically Distributed (non-IID) on the clients, which may bring the biases in the model training and cause possible accuracy degradation. To address this issue, this paper aims to propose a novel FL algorithm to alleviate the accuracy degradation caused by non-IID data at clients. Firstly, we observe that the clients with different degrees of non-IID data present heterogeneous weight divergence with the clients owning IID data. Inspired by this, we utilize weight divergence to recognize the non-IID degrees of clients. Then, we propose an efficient FL algorithm, named CSFedAvg, in which the clients with lower degree of non-IID data will be chosen to train the models with higher frequency. Finally, we conduct simulations using publicly-available datasets to train deep neural networks. Simulation results show that the proposed FL algorithm improves the training performance compared with existing FL protocol.

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Therefore, asynchronicity approaches aim to overcome the above-mentioned limitations through relaxing some of the synchronous federated learning assumptions. For example, the authors of [103] propose a Semi-Asynchronous Federated Averaging (SAFA) protocol that extends the FedAvg model. The solution capitalizes on asynchronous machine learning to reduce the impact of clients that drop offline (due to some power outage or low battery level) or are late in submitting the updated models (i.e., stragglers).…”

Section: A Communication Type Refers To the Manner In Which The Server And Clients Communicate The Training Model And Model Updatesmentioning

confidence: 99%

Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems

Wahab

Mourad

Otrok

et al. 2021

IEEE Commun. Surv. Tutorials

368

110

View full text Add to dashboard Cite

The communication and networking field is hungry for machine learning decision-making solutions to replace the traditional model-driven approaches that proved to be not rich enough for seizing the ever-growing complexity and heterogeneity of the modern systems in the field. Traditional machine learning solutions assume the existence of (cloud-based) central entities that are in charge of processing the data. Nonetheless, the difficulty of accessing private data, together with the high cost of transmitting raw data to the central entity gave rise to a decentralized machine learning approach called Federated Learning. The main idea of federated learning is to perform an on-device collaborative training of a single machine learning model without having to share the raw training data with any third-party entity. Although few survey articles on federated learning already exist in the literature, the motivation of this survey stems from three essential observations. The first one is the lack of a fine-grained multi-level classification of the federated learning literature, where the existing surveys base their classification on only one criterion or aspect. The second observation is that the existing surveys focus only on some common challenges, but disregard other essential aspects such as reliable client selection, resource management and training service pricing. The third observation is the lack of explicit and straightforward directives for researchers to help them design future federated learning solutions that overcome the state-of-the-art research gaps. To address these points, we first provide a comprehensive tutorial on federated learning and its associated concepts, technologies and learning approaches. We then survey and highlight the applications and future directions of federated learning in the domain of communication and networking. Thereafter, we design a three-level classification scheme that first categorizes the federated learning literature based on the high-level challenge that they tackle. Then, we classify each high-level challenge into a set of specific low-level challenges to foster a better understanding of the topic. Finally, we provide, within each low-level challenge, a fine-grained classification based on the technique used to address this particular challenge. For each category of high-level challenges, we provide a set of desirable criteria and future research directions that are aimed to help the research community design innovative and efficient future solutions. To the best of our knowledge, our survey is the most comprehensive in terms of challenges and techniques it covers and the most fine-grained in terms of the multi-level classification scheme it presents.

show abstract

“…Last but not least, we also want to note, FL itself is now far from its maturity, many important issues worth our study. Some of which might involve asynchronous or semi-asynchronous aggregation protocol [17], [18], incentive mechanism [19], [20] and security issues [21], etc. We look forward to more insightful and dedicated research into FL.…”

Section: Related Workmentioning

confidence: 99%

“…FL clients), then estimates the model exchange time with Eqs. (18) and (19) using historical information. Taking advantage of the observed context, availability as well as the estimation, the selection scheme for this round could be fetched by Algorithm 1.…”

mentioning

confidence: 99%

An Efficiency-Boosting Client Selection Scheme for Federated Learning With Fairness Guarantee

Huang

Lin

et al. 2021

IEEE Trans. Parallel Distrib. Syst.

Self Cite

147

View full text Add to dashboard Cite

The issue of potential privacy leakage during centralized AI's model training has drawn intensive concern from the public. A Parallel and Distributed Computing (or PDC) scheme, termed Federated Learning (FL), has emerged as a new paradigm to cope with the privacy issue by allowing clients to perform model training locally, without the necessity to upload their personal sensitive data. In FL, the number of clients could be sufficiently large, but the bandwidth available for model distribution and re-upload is quite limited, making it sensible to only involve part of the volunteers to participate in the training process. The client selection policy is critical to an FL process in terms of training efficiency, the final model's quality as well as fairness. In this paper, we will model the fairness guaranteed client selection as a Lyapunov optimization problem and then a C 2 MAB-based method is proposed for estimation of the model exchange time between each client and the server, based on which we design a fairness guaranteed algorithm termed RBCS-F for problem-solving. The regret of RBCS-F is strictly bounded by a finite constant, justifying its theoretical feasibility. Barring the theoretical results, more empirical data can be derived from our real training experiments on public datasets.

show abstract

SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead

Cited by 251 publications

References 8 publications

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems

An Efficiency-Boosting Client Selection Scheme for Federated Learning With Fairness Guarantee

Contact Info

Product

Resources

About