An Efficiency-Boosting Client Selection Scheme for Federated Learning With Fairness Guarantee

Huang, Tiansheng; Lin, Weiwei; Wu, Wentai; He, Ligang; Li, Keqin; Zomaya, Albert Y.

doi:10.1109/tpds.2020.3040887

Cited by 150 publications

(68 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The loss function considered, though, is simple linear regression and does not readily apply to neural network models. Stochastic optimization is also considered for FL in [27] and [28], but not to design an optimal device selection policy that guarantees convergence of non-convex loss functions like we do here.…”

Section: R Wmentioning

confidence: 99%

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

Perazzone¹,

Wang²,

Ji³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a useful tool in distributed machine learning that utilizes users' local datasets in a privacy-preserving manner. When deploying FL in a constrained wireless environment; however, training models in a time-efficient manner can be a challenging task due to intermittent connectivity of devices, heterogeneous connection quality, and non-i.i.d. data. In this paper, we provide a novel convergence analysis of nonconvex loss functions using FL on both i.i.d. and non-i.i.d. datasets with arbitrary device selection probabilities for each round. Then, using the derived convergence bound, we use stochastic optimization to develop a new client selection and power allocation algorithm that minimizes a function of the convergence bound and the average communication time under a transmit power constraint. We find an analytical solution to the minimization problem. One key feature of the algorithm is that knowledge of the channel statistics is not required and only the instantaneous channel state information needs to be known. Using the FEMNIST and CIFAR-10 datasets, we show through simulations that the communication time can be significantly decreased using our algorithm, compared to uniformly random participation.

show abstract

Section: R Wmentioning

confidence: 99%

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

Perazzone¹,

Wang²,

Ji³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We follow a more practical approach based on the observation that membership-related information is only sensitive in the last DNN layer, making it vulnerable to MIAs as indicated in previous research [47,49,52,59]. [21,53].…”

Section: Classifiermentioning

confidence: 99%

PPFL

Haddadi

Katevas

et al. 2021

Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services

139

View full text Add to dashboard Cite

We propose and implement a Privacy-preserving Federated Learning () framework for mobile systems to limit privacy leakages in federated learning. Leveraging the widespread presence of Trusted Execution Environments (TEEs) in high-end and mobile devices, we utilize TEEs on clients for local training, and on servers for secure aggregation, so that model/gradient updates are hidden from adversaries. Challenged by the limited memory size of current TEEs, we leverage greedy layer-wise training to train each model's layer inside the trusted area until its convergence. The performance evaluation of our implementation shows that can significantly improve privacy while incurring small system overheads at the client-side. In particular, can successfully defend the trained model against data reconstruction, property inference, and membership inference attacks. Furthermore, it can achieve comparable model utility with fewer communication rounds (0.54×) and a similar amount of network traffic (1.002×) compared to the standard federated learning of a complete model. This is achieved while only introducing up to ∼15% CPU time, ∼18% memory usage, and ∼21% energy consumption overhead in 's client-side. CCS CONCEPTS• Security and privacy → Privacy protections; Distributed systems security; • Computing methodologies → Distributed algorithms.

show abstract

“…[28] designs a client scheduling problem and provides a MAB-based framework for FL training without knowing the wireless channel state information and the dynamic usage of local computing resources. In order to minimize the latency, [29] models fair-guaranteed client selection as a Lyapunov optimization problem and presents a policy based on CC-MAB to estimate the model transmission time. A multi-agent MAB algorithm is developed to minimize the FL training latency over wireless channels, constrained by training performance as well as each client's differential privacy requirement in [30].…”

Section: Related Workmentioning

confidence: 99%

Context-Aware Online Client Selection for Hierarchical Federated Learning

Qu¹,

Duan²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) has been considered as an appealing framework to tackle data privacy issues of mobile devices compared to conventional Machine Learning (ML). Using Edge Servers (ESs) as intermediaries to perform model aggregation in proximity can reduce the transmission overhead, and it enables great potentials in low-latency FL, where the hierarchical architecture of FL (HFL) has been attracted more attention. Designing a proper client selection policy can significantly improve training performance, and it has been extensively used in FL studies. However, to the best of our knowledge, there are no studies focusing on HFL. In addition, client selection for HFL faces more challenges than conventional FL, e.g., the time-varying connection of client-ES pairs and the limited budget of the Network Operator (NO). In this paper, we investigate a client selection problem for HFL, where the NO learns the number of successful participating clients to improve the training performance (i.e., select as many clients in each round) as well as under the limited budget on each ES. An online policy, called Context-aware Online Client Selection (COCS), is developed based on Contextual Combinatorial Multi-Armed Bandit (CC-MAB). COCS observes the side-information (context) of local computing and transmission of client-ES pairs and makes client selection decisions to maximize NO's utility given a limited budget. Theoretically, COCS achieves a sublinear regret compared to an Oracle policy on both strongly convex and non-convex HFL. Simulation results also support the efficiency of the proposed COCS policy on real-world datasets.

show abstract

An Efficiency-Boosting Client Selection Scheme for Federated Learning With Fairness Guarantee

Cited by 150 publications

References 34 publications

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

PPFL

Context-Aware Online Client Selection for Hierarchical Federated Learning

Contact Info

Product

Resources

About