Online Client Selection for Asynchronous Federated Learning With Fairness Consideration

Zhu, Hongbin; Zhou, Yong; Qian, Hua; Shi, Yuanming; Chen, Xu; Yang, Yang

doi:10.1109/twc.2022.3211998

Cited by 33 publications

(15 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Players should examine the surroundings to gain more knowledge on each training round, recognize activities that boost the chance of achieving higher rewards, or exploit existing knowledge to execute the actions that reasonably worked in the past. This method has been used to design client scheduling [ 1 , 7 , 59 ] or in the CS process [ 91 , 92 , 93 ]. Three main categories arise from the proposed procedures to decrease the training latency in FL: Update compression (quantizing gradient is a solution for efficient communication).…”

Section: Pros and Cons Of Different Cs Methodsmentioning

confidence: 99%

A Comprehensive Overview of IoT-Based Federated Learning: Focusing on Client Selection Methods

Khajehali,

Yan,

Chow

et al. 2023

Sensors

View full text Add to dashboard Cite

The integration of the Internet of Things (IoT) with machine learning (ML) is revolutionizing how services and applications impact our daily lives. In traditional ML methods, data are collected and processed centrally. However, modern IoT networks face challenges in implementing this approach due to their vast amount of data and privacy concerns. To overcome these issues, federated learning (FL) has emerged as a solution. FL allows ML methods to achieve collaborative training by transferring model parameters instead of client data. One of the significant challenges of federated learning is that IoT devices as clients usually have different computation and communication capacities in a dynamic environment. At the same time, their network availability is unstable, and their data quality varies. To achieve high-quality federated learning and handle these challenges, designing the proper client selection process and methods are essential, which involves selecting suitable clients from the candidates. This study presents a comprehensive systematic literature review (SLR) that focuses on the challenges of client selection (CS) in the context of federated learning (FL). The objective of this SLR is to facilitate future research and development of CS methods in FL. Additionally, a detailed and in-depth overview of the CS process is provided, encompassing its abstract implementation and essential characteristics. This comprehensive presentation enables the application of CS in diverse domains. Furthermore, various CS methods are thoroughly categorized and explained based on their key characteristics and their ability to address specific challenges. This categorization offers valuable insights into the current state of the literature while also providing a roadmap for prospective investigations in this area of research.

show abstract

Section: Pros and Cons Of Different Cs Methodsmentioning

confidence: 99%

A Comprehensive Overview of IoT-Based Federated Learning: Focusing on Client Selection Methods

Khajehali,

Yan,

Chow

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…In contrast to synchronous FL, the asynchronous scheme leads to faster convergence under unstable networks especially with millions of devices [28]. An increasing number of asynchronous FL works have been published in recent years, with focuses on client selection [17,24,29,63,64], weight aggregation [52,53,60] and transmission scheduling [35]. Semi-asynchronous mechanisms are developed to aggregate buffered updates [14,16,45,54,61].…”

Section: Related Workmentioning

confidence: 99%

“…Prioritizing the nodes with high loss or large gradients' norm is the state-of-the-art approach for asynchronous FL [27]. We did not compare with [64] as their algorithm depends on completely different metrics.…”

Section: Experiments Setupmentioning

confidence: 99%

Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks

Yu¹,

Cherkasova²,

Vardhan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) has gained increasing interest in recent years as a distributed on-device learning paradigm. However, multiple challenges remain to be addressed for deploying FL in real-world Internet-of-Things (IoT) networks with hierarchies. Although existing works have proposed various approaches to account data heterogeneity, system heterogeneity, unexpected stragglers and scalibility, none of them provides a systematic solution to address all of the challenges in a hierarchical and unreliable IoT network. In this paper, we propose an asynchronous and hierarchical framework (Async-HFL) for performing FL in a common three-tier IoT network architecture. In response to the largely varied delays, Async-HFL employs asynchronous aggregations at both the gateway and the cloud levels thus avoids long waiting time. To fully unleash the potential of Async-HFL in converging speed under system heterogeneities and stragglers, we design device selection at the gateway level and device-gateway association at the cloud level. Device selection chooses edge devices to trigger local training in real-time while device-gateway association determines the network topology periodically after several cloud epochs, both satisfying bandwidth limitation. We evaluate Async-HFL's convergence speedup using large-scale simulations based on ns-3 and a network topology from NYCMesh. Our results show that Async-HFL converges 1.08-1.31x faster in wall-clock time and saves up to 21.6% total communication cost compared to state-of-the-art asynchronous FL algorithms (with client selection). We further validate Async-HFL on a physical deployment and observe robust convergence under unexpected stragglers.

show abstract

“…28 Zhu et al 29 have proposed AFL to deal with stragglers. The paper 29 advocates an asynchronous FL framework with adaptive client selection for training latency minimization, client availability and long-term fairness. Similarly, Ma et al 20 have proposed semi-asynchronous FL framework(FedSA) for distributed edge computing.…”

Section: Related Workmentioning

confidence: 99%

“…In such a network, devices are bound to fail/struggle, needing different scheduling and aggregation strategies 28 . Zhu et al 29 have proposed AFL to deal with stragglers. The paper 29 advocates an asynchronous FL framework with adaptive client selection for training latency minimization, client availability and long‐term fairness.…”

Section: Related Workmentioning

confidence: 99%

FIDEL: Fog integrated federated learning framework to train neural networks

Kumar,

Srirama

2023

Softw Pract Exp

View full text Add to dashboard Cite

Technological advancement in the digital era has continued to produce voluminous amounts of data through various devices. Even though data is produced distributively, it needs to be accumulated centrally for processing, analysis, and knowledge extraction that faces several challenges such as bandwidth, latency, congestion, privacy, and security. Fog computing paradigm addresses some of these issues, and can be used as a distributed data processing unit. Federated learning trains a shared model over distributed nodes. However, a fog node can not process continuously growing data due to computational limitations. In this paper, we propose FIDEL: a fog integrated federated learning framework for neural network training using resource‐constrained devices. The federation of resource‐constrained Internet of Things (IoT) devices creates a shared global model trained on local data, which is generalized on the unseen dataset for prediction/inferences. We have also designed an online training scheme to process continuous data with limited compute resources. The FIDEL supports both synchronous and asynchronous federate learning that empowers resource‐constrained devices to train machine learning models. To test the learning capabilities of the FIDEL, we have trained three neural networks (i) Shallow network; (ii) Deep Network; (iii) Convolutional Neural Network (CNN) models for human position detection in industrial IoT setup on rapidly changing datasets. The experimental results show that the framework can learn input–output relationships with significantly high accuracy. The overall system efficiency of the framework is reasonable in terms of latency and memory usage for resource‐constrained devices.

show abstract

Online Client Selection for Asynchronous Federated Learning With Fairness Consideration

Cited by 33 publications

References 36 publications

A Comprehensive Overview of IoT-Based Federated Learning: Focusing on Client Selection Methods

A Comprehensive Overview of IoT-Based Federated Learning: Focusing on Client Selection Methods

Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks

FIDEL: Fog integrated federated learning framework to train neural networks

Contact Info

Product

Resources

About