BalanceFL: Addressing Class Imbalance in Long-Tail Federated Learning

Shuai, Xian; Shen, Yuh Chiang; Jiang, Siyang; Yan, Zhenyu; Xing, Guoliang

doi:10.1109/ipsn54338.2022.00029

Cited by 32 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, previous studies specifically take into account the label skew non-IID when each client has a fixed number of classes (label size imbalance [8], [9], [28], [43]- [45]) or when the number of samples of a certain class is distributed to clients using the power-law or Dirichlet distribution (label distribution imbalance [13], [27], [39], [46]). Recently, some works consider the non-IID scenario which is more close to the real-world data such as the numbers of classes are often highly imbalanced [29], or following the cluster-skew non-IID distribution [30], [31]. Especially, cluster-skew is firstly introduced by [30] where there exists a data correlation between clients.…”

Section: Related Workmentioning

confidence: 99%

“…In particular, non-IID data may slow down the model convergence, destabilize local training at clients, and degrade model accuracy in consequence [15]- [19]. Numerous efforts have been devoted to overcoming the non-IID issue, which may be classified into two primary categories: (i) reduce the impact of non-IID data by optimizing the aggregation [10], [20], [21] or by optimizing the method to select the client each round [22]- [24] on the server-side, and (ii) enhancing training on the client side [9], [25]- [29]. However, current research on non-IID faces the two critical issues as follows.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Teachers’ awareness of happy schools with UNESCO items: a case study of a primary school in Hanoi, Vietnam

Nguyen¹,

Le²,

Nguyen³

et al. 2022

VMOST

View full text Add to dashboard Cite

This article provides the practice of primary teachers’ awareness of happy schools through a survey questionnaire based on 22 items from the UNESCO criteria on happy schools. Their opinions were shared after a number of workshop activities that teach the practice of happy schools in Quan Hoa primary school, Cau Giay district, Hanoi. To add more research detail, the authors performed an in-depth interview with the vice principal. The research results showed that primary teachers’ awareness was at a good level (100% agreed). Twenty-seven teachers in this workshop gave a detailed and reflective explanation of happy schools. Then, the authors provided a number of suggestions to spread the idea of happy schools in practice, within primary schools in particular, and for all schools in general, according to the UNESCO items, which are on the move in regulations and curricula at the Bureau of Education and Training of Hanoi these days, especially in the context of the 2018 National Education Curriculum implemented since the 2020-2021 school year.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Teachers’ awareness of happy schools with UNESCO items: a case study of a primary school in Hanoi, Vietnam

Nguyen¹,

Le²,

Nguyen³

et al. 2022

VMOST

View full text Add to dashboard Cite

show abstract

“…The Astraea offers to solve the global imbalance problem by z-score-based data augmentation and down sampling as well as a mediator for rescheduling the attendance of collaborators by exploiting the Kullback-Leibler divergence of their distributions [18]. Shuai et al [19] offered BalanceFL method which aims to solve the local and global data imbalance by updating the locally trained models as it is trained on uniform data. In the BalanceFL study, the local imbalance problems are defined as causing from two reasons such as data amount and class missing, which are solved respectively by inter-class balancing and knowledge inheritance techniques [19].…”

Section: Fl Applications On Imbalanced Datamentioning

confidence: 99%

“…Shuai et al [19] offered BalanceFL method which aims to solve the local and global data imbalance by updating the locally trained models as it is trained on uniform data. In the BalanceFL study, the local imbalance problems are defined as causing from two reasons such as data amount and class missing, which are solved respectively by inter-class balancing and knowledge inheritance techniques [19].…”

Section: Fl Applications On Imbalanced Datamentioning

confidence: 99%

Collaborative Colorectal Cancer Classification on Highly Class Imbalanced Data Setting via Federated Neural Style Transfer Based Data Augmentation

Nergiz¹

2022

View full text Add to dashboard Cite

The deep learning algorithms achieved promising results in the computational pathology in recent decade but the high data demand of the deep learning algorithms get stuck in the multi-institutional data collaborations. The federated learning is a novel concept, which proposes to train the models of the different sites collaboratively via an orchestrating server without leaking private data. However, the imbalanced data distributions are challenging for federated learning and result in performance decrease and destabilization. In this study, the federated version of the neural style transfer algorithm, which was offered by Gatys et al. is proposed as a data augmentation method on the highly class imbalanced configuration of Chaoyang colorectal cancer imaging dataset. The proposed method works by firstly selecting characteristic style images and then generating the gram style matrices on the local sites and then transferring them to the other imbalanced sites by not leaking any private data. The proposed method contributed the ACC, F1 Score and AUC results of pure FL by 22.07%, 42.51% and 9.65% using only 20 images for content and 5 images for style. Additionally, the experiments having different content and style numbers achieved the satisfactory and consisting results.

show abstract

“…Accompanied by the prospect of FL, it faces the challenge of data heterogeneity, which is a universal issue arising from disparities in data distribution among clients due to data source differences between clients (Karimireddy et al 2020;. Furthermore, real-world data frequently exhibits the long-tailed phenomenon (Shuai et al 2022), that is, the class distribution is imbalanced, where the head classes have a large number of samples while the tail classes have a small number of samples. Longtailed data makes client-side models biased toward the head class (Shang et al 2022).…”

Section: Introductionmentioning

confidence: 99%

CLIP-Guided Federated Learning on Heterogeneity and Long-Tailed Data

Shi,

Zheng,

Yin

et al. 2024

AAAI

View full text Add to dashboard Cite

Federated learning (FL) provides a decentralized machine learning paradigm where a server collaborates with a group of clients to learn a global model without accessing the clients' data. User heterogeneity is a significant challenge for FL, which together with the class-distribution imbalance further enhances the difficulty of FL. Great progress has been made in large vision-language models, such as Contrastive Language-Image Pre-training (CLIP), which paves a new way for image classification and object recognition. Inspired by the success of CLIP on few-shot and zero-shot learning, we use CLIP to optimize the federated learning between server and client models under its vision-language supervision. It is promising to mitigate the user heterogeneity and class-distribution balance due to the powerful cross-modality representation and rich open-vocabulary prior knowledge. In this paper, we propose the CLIP-guided FL (CLIP2FL) method on heterogeneous and long-tailed data. In CLIP2FL, the knowledge of the off-the-shelf CLIP model is transferred to the client-server models, and a bridge is built between the client and server. Specifically, for client-side learning, knowledge distillation is conducted between client models and CLIP to improve the ability of client-side feature representation. For server-side learning, in order to mitigate the heterogeneity and class-distribution imbalance, we generate federated features to retrain the server model. A prototype contrastive learning with the supervision of the text encoder of CLIP is introduced to generate federated features depending on the client-side gradients, and they are used to retrain a balanced server classifier. Extensive experimental results on several benchmarks demonstrate that CLIP2FL achieves impressive performance and effectively deals with data heterogeneity and long-tail distribution. The code is available at https://github.com/shijiangming1/CLIP2FL.

show abstract

BalanceFL: Addressing Class Imbalance in Long-Tail Federated Learning

Cited by 32 publications

References 22 publications

Teachers’ awareness of happy schools with UNESCO items: a case study of a primary school in Hanoi, Vietnam

Teachers’ awareness of happy schools with UNESCO items: a case study of a primary school in Hanoi, Vietnam

Collaborative Colorectal Cancer Classification on Highly Class Imbalanced Data Setting via Federated Neural Style Transfer Based Data Augmentation

CLIP-Guided Federated Learning on Heterogeneity and Long-Tailed Data

Contact Info

Product

Resources

About