FedSA: A staleness-aware asynchronous Federated Learning algorithm with non-IID data

Chen, Ming; Mao, Bingcheng; Ma, Tianyi

doi:10.1016/j.future.2021.02.012

Cited by 55 publications

(18 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A stale model may marginally contribute to the global model, but cause large resource waste and time consumption without timely terminating model training and uploading. To address this issue, FedSA [45], a staleness-aware asynchronous FL algorithm sets a staleness threshold for each participating client based on its computing speed. However, this approach ignores the communication cost and the training time in the whole process.…”

Section: Model Staleness Controlmentioning

confidence: 99%

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

Wu¹,

Chen²,

Ouyang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a promising paradigm that enables collaboratively learning a shared model across massive clients while keeping the training data locally. However, for many existing FL systems, clients need to frequently exchange model parameters of large data size with the remote cloud server directly via wide-area networks (WAN), leading to significant communication overhead and long transmission time. To mitigate the communication bottleneck, we resort to the hierarchical federated learning paradigm of HiFL, which reaps the benefits of mobile edge computing and combines synchronous client-edge model aggregation and asynchronous edge-cloud model aggregation together to greatly reduce the traffic volumes of WAN transmissions. Specifically, we first analyze the convergence bound of HiFL theoretically and identify the key controllable factors for model performance improvement. We then advocate an enhanced design of HiFlash by innovatively integrating deep reinforcement learning based adaptive staleness control and heterogeneity-aware client-edge association strategy to boost the system efficiency and mitigate the staleness effect without compromising model accuracy. Extensive experiments corroborate the superior performance of HiFlash in model accuracy, communication reduction, and system efficiency.

show abstract

Section: Model Staleness Controlmentioning

confidence: 99%

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

Wu¹,

Chen²,

Ouyang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…for different i th and j th client. Moreover, some research [17] also considers that the data are non-i.i.d. if the expectation of local gradients and global gradients are different:…”

Section: Recent Communication Challenges In Flmentioning

confidence: 99%

Towards Efficient Communications in Federated Learning: A Contemporary Survey

Zhen¹,

Mao²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

In the traditional distributed machine learning scenario, the user's private data is transmitted between nodes and a central server, which results in great potential privacy risks. In order to balance the issues of data privacy and joint training of models, federated learning (FL) is proposed as a special distributed machine learning with a privacy protection mechanism, which can realize multiparty collaborative computing without revealing the original data. However, in practice, FL faces many challenging communication problems. This review aims to clarify the relationship between these communication problems, and focus on systematically analyzing the research progress of FL communication work from three perspectives: communication efficiency, communication environment, and communication resource allocation. Firstly, we sort out the current challenges existing in the communications of FL. Secondly, we have compiled articles related to FL communications, and then describe the development trend of the entire field guided by the logical relationship between them. Finally, we point

show abstract

“…The authors of [106] proposed a method called FedBN for normalizing the client data and shifting the features of the data before aggregation. The authors of [107] and [108] proposed a method called FedSA [107] and FedUFO [108] to manage unstable, unreliable, inconsistentency, and feature divergence problems in distributed non-IID. Using these two methods, the learning performance is increased in federated learning.…”

Section: Cifar-10mentioning

confidence: 99%

Federated Learning Design and FunctionalModels: Survey

John

Utomo

Rouniyar

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning is a multiple device collaboration setup designed tosolve machine learning problems under framework for aggregation andknowledge transfer in distributed local data. This distributed modelensures the privacy of data at each local node. Owing to its relevance,there has been extensive research activities and outcomes in federatedlearning with expanded applicability to different areas by the researchcommunity. As such, there is a vast research archive made available by thecommunity with research work and articles related to the various aspectsof federated learning such as applications, challenges, privacy, function-alities, and design. With respect to the function and design of federatedlearning, client selection, aggregation, knowledge transfer, managementof distributed data (Non-IID), Incentive of data and communication costare of paramount importance. Any effective design of federated learningrequires these aspects to be well considered.There are numerous surveyarticles found among the available literature that focus on its applica-tion and challenges, opportunities, data privacy and protection, as wellas on federated learning on internet of things, federated learning on edgecomputing, etc.1 In this paper, a review of the available literature on the various ele-ments of design and functionalities in federated learning has been carriedout with an aim to lay emphasis on the important challenges andresearch opportunities. More specifically, this work has endeavored tounderstand and summarize the various functional methods available,along with their techniques and goals. Additionally, it has strived toget a bird’s eye view of how various functions and designs of feder-ated learning have been used in applications, and how it has helpeduncover challenges and promising research directions for the future.

show abstract

FedSA: A staleness-aware asynchronous Federated Learning algorithm with non-IID data

Cited by 55 publications

References 1 publication

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

Towards Efficient Communications in Federated Learning: A Contemporary Survey

Federated Learning Design and FunctionalModels: Survey

Contact Info

Product

Resources

About