Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Abstract

Federated Learning (FL) has recently attracted considerable attention in internet of things, due to its capability of enabling mobile clients to collaboratively learn a global prediction model without sharing their privacy-sensitive data to the server. Despite its great potential, a main challenge of FL is that the training data are usually non-Independent, Identically Distributed (non-IID) on the clients, which may bring the biases in the model training and cause possible accuracy degradation. To address this issue, this paper aims to propose a novel FL algorithm to alleviate the accuracy degradation caused by non-IID data at clients. Firstly, we observe that the clients with different degrees of non-IID data present heterogeneous weight divergence with the clients owning IID data. Inspired by this, we utilize weight divergence to recognize the non-IID degrees of clients. Then, we propose an efficient FL algorithm, named CSFedAvg, in which the clients with lower degree of non-IID data will be chosen to train the models with higher frequency. Finally, we conduct simulations using publicly-available datasets to train deep neural networks. Simulation results show that the proposed FL algorithm improves the training performance compared with existing FL protocol.

Industrial Internet Federated Learning Driven by IoT Equipment ID and Blockchain

“…Furthermore, it can carry out high-efficiency machine learning among multiple participants or multiple computing nodes. Zhang et al [19] made a comprehensive review of recent research and achievements in federal learning and presented future development trends. First, data islands and privacy protection are described to introduce the background of federated learning, and the connotation and mechanism of federated learning are outlined.…”

WCL: Client Selection in Federated Learning with a Combination of Model Weight Divergence and Client Training Loss for Internet Traffic Classification

“…Specifically, in the client selection algorithm, the power-ofchoice framework improves the convergence speed of the model and the effect is obvious. Zhang et al [6] mentioned the influence of the degree of nonindependent (non-IID) of client data on FL model training. The highly heterogeneous data distribution caused by non-IID data will bring bias in model training and may lead to a decrease in the accuracy of the FL model.…”

“…The highly heterogeneous data distribution caused by non-IID data will bring bias in model training and may lead to a decrease in the accuracy of the FL model. Therefore, Zhang et al [6] proposed a new FL method CSFedAvg. The CSFedAvg method uses the weight divergence to identify the non-IID degrees of clients, and CSFedAvg selects client update data with lower degree of non-IID according to the weight divergence to train the global model.…”

“…Therefore, how to alleviate the bias of high heterogeneous data in global model training in FL has become a hot research topic. The existing works mainly adopt to weighted averaging the local model parameters [6] or selecting the clients according to the same sign counts between the local models and global model [7], which can hardly eliminate the high heterogeneity clients in model aggregation.…”

Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies

“…To position existing research attempts in optimizing the time-to-accuracy performance in FL, we propose a layered approach that categorizes them by the training phases at which they take effect: selection, configuration, or reporting ( § III). For the selection phase where the server chooses clients for participation, there are mainly two lines of optimization efforts: (1) one focuses on prioritizing clients either with high statistical utility or system utility [17], [18], and (2) the other explicitly considers both utilities and works out more informed solutions in response to client dynamics in reality [19], [20]. As for the configuration phase where the server sends the global model to selected clients with auxiliary configuration information and clients perform local training, we sort out four lines of work: (1) the first two lines advocate mitigating the communication cost by reducing model size [21]- [34] and decreasing synchronization frequency [35]- [39]; while (2) the last two lines of minimize computational overhead by shortening training latency in a round [40]- [44] as well as bringing down the number of training rounds [45]- [50].…”