Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data

Yang, Chengxu; Wang, Qipeng; Xu, Mengwei; Chen, Zhenpeng; Bian, Kaigui; Liu, Yunxin; Liu, Xuanzhe

doi:10.1145/3442381.3449851

Cited by 95 publications

(72 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Celeba dataset contains face attributes of 915 users with 19,923 samples. We use CNN models for both datasets as in previous work [68]. • Natural Language Processing (NLP): We evaluate Fed-Balancer on two NLP tasks each on different dataset: next-word prediction on Reddit [10] dataset and nextcharacter prediction on Shakespeare [57] dataset.…”

Section: Methodsmentioning

confidence: 99%

“…Method. We first ran FedAvg+1𝑇 on five datasets until convergence, with the number of rounds that are suggested by previous works [10,11,61,68]: 1000, 100, 600, 40, and 300 rounds for FEMNIST, Celeba, Reddit, Shakespeare, and UCI-HAR. Based on the user trace data of FLASH, we measured the wall clock time which FedAvg+1𝑇 ran for each dataset, and ran experiments with other baselines and FedBalancer until the same wall clock time.…”

Section: Methodsmentioning

confidence: 99%

“…Other configurations. As in previous study [68], we use batch size of 100 for Shakespeare and 10 for rest of the datasets. We select 100 clients at each round for datasets with more than 500 users, and otherwise select 10 users for Shakespeare and 5 users for UCI-HAR.…”

Section: Methodsmentioning

confidence: 99%

“…A common practice for such an issue is to set a deadline for a round duration and drop clients who fail to send the model updates before the deadline [2,48]. However, this results in less contribution from clients with low-end devices, which could result in delayed convergence or biased model training [41,68].…”

Section: Background and Motivation 21 Federated Learningmentioning

confidence: 99%

“…In a training round of FL, the computation time has been shown to be the bottleneck [51,64,68]. As the amount of client computation changes with the sample selection of Fed-Balancer, finding an optimal deadline could be an important problem in achieving high time-to-accuracy performance.…”

Section: Motivational Studymentioning

confidence: 99%

See 4 more Smart Citations

FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients

Shin¹,

Li²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Federated Learning (FL) trains a machine learning model on distributed clients without exposing individual data. Unlike centralized training that is usually based on carefullyorganized data, FL deals with on-device data that are often unfiltered and imbalanced. As a result, conventional FL training protocol that treats all data equally leads to a waste of local computational resources and slows down the global learning process. To this end, we propose FedBalancer, a systematic FL framework that actively selects clients' training samples. Our sample selection strategy prioritizes more "informative" data while respecting privacy and computational capabilities of clients. To better utilize the sample selection to speed up global training, we further introduce an adaptive deadline control scheme that predicts the optimal deadline for each round with varying client train data. Compared with existing FL algorithms with deadline configuration methods, our evaluation on five datasets from three different domains shows that FedBalancer improves the time-to-accuracy performance by 1.22∼4.62× while improving the model accuracy by 1.0∼3.3%. We also show that FedBalancer is readily applicable to other FL approaches by demonstrating that FedBalancer improves the convergence speed and accuracy when operating jointly with three different FL algorithms.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Background and Motivation 21 Federated Learningmentioning

confidence: 99%

Section: Motivational Studymentioning

confidence: 99%

See 3 more Smart Citations

FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients

Shin¹,

Li²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Decentralized federated meta‐learning framework for few‐shot multitask learning

Wang

et al. 2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

Federated learning is increasingly attractive, however as the number of training samples on a single device is too small and the training tasks of the devices are different, it faces the few-shot multitask learning problem. Moreover, federated learning frameworks are usually vulnerable to malicious attacks of the central server and diverse clients.To address these problems, we propose a decentralized federated meta-learning framework (DFMLF) for fewshot multitask learning. In DFMLF, the devices take the rapid adaptation as objective and learn the metaknowledge shared by tasks to deal with the few-shot multitask problem. In addition, DFMLF conducts crossvalidation and secure aggregation mechanism by a small number of committee nodes, which not only eliminates the central server to avoid the security risks brought by the malicious central server, but also avoids the attack of malicious devices. Moreover, to address the extra communication cost brought by the committee strategy, we propose a communication-efficient method to make the training and aggregation carried out in parallel.We conduct extensive experiments based on real-world data sets, and the experimental results demonstrate the effectiveness, robustness, and efficiency of our framework.

show abstract