Contribution‐based Federated Learning client selection

Abstract

Federated Learning (FL), as a privacy-preserving machine learning paradigm, has been thrusted into the limelight. As a result of the physical bandwidth constraint, only a small number of clients are selected for each round of FL training. However, existing client selection solutions (e.g., the vanilla random selection) typically ignore the heterogeneous data value of the clients. In this paper, we propose the contribution-based selection algorithm (Contribution-Based Exponentialweight algorithm for Exploration and Exploitation, CBE3), which dynamically updates the selection weights according to the impact of clients' data. As a novel component of CBE3, a scaling factor, which helps maintain a good balance between global model accuracy and convergence speed, is proposed to improve the algorithm's adaptability. Theoretically, we proved the regret bound of the proposed CBE3 algorithm, which demonstrates performance gaps between the CBE3 and the optimal choice. Empirically, extensive experiments conducted on Non-Independent Identically Distributed data demonstrate the superior performance of CBE3with up to 10% accuracy improvement compared with K-Center and Greedy and up to 100% faster convergence compared with the Random algorithm.

A Comprehensive Overview of IoT-Based Federated Learning: Focusing on Client Selection Methods

“…In recent years, client selection (CS) methods have been introduced as one of the essential solutions to alleviate the above challenges [ 33 , 34 , 35 ]. Overall, the server evaluates a client’s performance based on information from the local models it receives [ 36 ].…”

“…Overall, the server evaluates a client’s performance based on information from the local models it receives [ 36 ]. Due to bandwidth limitations [ 37 ] and the availability of many clients [ 34 ], a selected subset of them can take part in the process at each training round [ 37 ]. It should be noted that clients usually have significant differences in terms of resource constraints, heterogeneous hardware conditions, and data resources in many procedures [ 38 ].…”

“…The CS process actively selects participating clients based on predefined criteria for each training round. Researchers have demonstrated that different criteria can be adopted to achieve objectives like fast convergence, low communication costs, optimal final model, and maximizing the overall performance of a model [ 1 , 34 , 35 ]. For example, the authors in [ 1 ] show that compared with the vanilla CS algorithm for FL, given the same number of rounds, their proposed CS solution (RBCS-F) decreased the training time.…”

“…It is important to know whether CS can enhance the convergence rate, achieve more stable training, and improve final accuracy. In relation to these considerations, CS can be an effective solution for FL optimization [ 34 , 41 ]. Under large-scale FL scenarios, finding a suitable CS technique requires a massive search area with a non-polynomial time complexity that cannot be performed in real-time.…”

“…Mishandling these challenges can adversely affect the performance of FL. Hence, CS methods are used to help solve these challenges [ 6 , 34 ]. Employing an effective FL CS method, handling the heterogeneity of data and clients, reducing training overheads, guaranteeing privacy, efficient communication, strengthening robustness, and improving model accuracy can be achieved.…”