FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

Li, Xiaoxiao; Jiang, Meirui; Zhang, Xiaofei; Kamp, Michael; Dou, Qi

doi:10.48550/arxiv.2102.07623

Cited by 60 publications

(93 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Communication efficiency, system heterogeneity, statistical heterogeneity, and privacy are all major issues in FL [16]. To reduce the communication costs in FL, some studies propose using data compression techniques [3,17,18], adding regularization terms for local optimization [6,19], and developing FL counterparts of Batch Normalization [7,20,21]. Moreover, the use of local momentum and global momentum [22] have been shown to facilitate faster convergence.…”

Section: Related Workmentioning

confidence: 99%

SemiFL: Communication Efficient Semi-Supervised Federated Learning with Unlabeled Clients

Diao¹,

Ding²,

Tarokh³

2021

Preprint

View full text Add to dashboard Cite

Federated Learning allows training machine learning models by using the computation and private data resources of a large number of distributed clients such as smartphones and IoT devices. Most existing works on Federated Learning (FL) assume the clients have ground-truth labels. However, in many practical scenarios, clients may be unable to label task-specific data, e.g., due to lack of expertise. In this work, we consider a server that hosts a labeled dataset, and wishes to leverage clients with unlabeled data for supervised learning. We propose a new Federated Learning framework referred to as SemiFL in order to address the problem of Semi-Supervised Federated Learning (SSFL). In SemiFL, clients have completely unlabeled data, while the server has a small amount of labeled data. SemiFL is communication efficient since it separates the training of server-side supervised data and client-side unsupervised data. We demonstrate various efficient strategies of SemiFL that enhance learning performance. Extensive empirical evaluations demonstrate that our communication efficient method can significantly improve the performance of a labeled server with unlabeled clients. Moreover, we demonstrate that SemiFL can outperform many existing FL results trained with fully supervised data, and perform competitively with the state-of-the-art centralized Semi-Supervised Learning (SSL) methods. For instance, in standard communication efficient scenarios, our method can perform 93% accuracy on the CIFAR10 dataset with only 4000 labeled samples at the server. Such accuracy is only 2% away from the result trained from 50000 fully labeled data, and it improves about 30% upon existing SSFL methods in the communication efficient setting.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

SemiFL: Communication Efficient Semi-Supervised Federated Learning with Unlabeled Clients

Diao¹,

Ding²,

Tarokh³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, in the federated setting, data is stored locally and cannot be shared. Recently, federated batch normalization [3] and federated adversarial domain adaptation [2], [9] have been proposed to deal with DA under the privacy-preserving requirement. The work by Li et al feature shift, i.e., the deviation in feature space, using batch normalization before averaging the local models.…”

Section: B Domain Adaptationmentioning

confidence: 99%

“…To cope with such diversity, recent works [2], [3] have proposed to integrate Unsupervised Domain Adaptation (UDA) into the FL framework. UDA methods force the model to learn domain-agnostic features through adversarial learning [2] or a specific type of batch normalization [3]. In this work, we follow an UDA adversarial approach to handle non-IID data.…”

mentioning

confidence: 99%

Memory-aware curriculum federated learning for breast cancer classification

Jiménez-Sánchez¹,

Tardy²,

Ballester³

et al. 2021

Preprint

View full text Add to dashboard Cite

For early breast cancer detection, regular screening with mammography imaging is recommended. Routinary examinations result in datasets with a predominant amount of negative samples. A potential solution to such class-imbalance is joining forces across multiple institutions. Developing a collaborative computer-aided diagnosis system is challenging in different ways. Patient privacy and regulations need to be carefully respected. Data across institutions may be acquired from different devices or imaging protocols, leading to heterogeneous non-IID data. Also, for learning-based methods, new optimization strategies working on distributed data are required. Recently, federated learning has emerged as an effective tool for collaborative learning. In this setting, local models perform computation on their private data to update the global model. The order and the frequency of local updates influence the final global model. Hence, the order in which samples are locally presented to the optimizers plays an important role. In this work, we define a memoryaware curriculum learning method for the federated setting.Our curriculum controls the order of the training samples paying special attention to those that are forgotten after the deployment of the global model. Our approach is combined with unsupervised domain adaptation to deal with domain shift while preserving data privacy. We evaluate our method with three clinical datasets from different vendors. Our results verify the effectiveness of federated adversarial learning for the multi-site breast cancer classification. Moreover, we show that our proposed memoryaware curriculum method is beneficial to further improve classification performance. Our code is publicly available at: https://github.com/ameliajimenez/curriculum-federatedlearning.

show abstract

“…Solutions to overcome the non-iid challenge thus need to be developed, e.g., creating an additional subset of datasets to allocate fairly among clients [37], aiming to ensure efficient data training in FL-based smart healthcare. Another promising approach is to implement the feature shift among heterogeneous clients [127], by using local batch normalization to adjust the feature distributions at the client side before averaging the local models. Quantitative metrics are needed to assess non-iid data in the FL-based smart healthcare sector, such as standard deviation, precision, and accuracy with respect to label/feature distribution skew and homogeneous partitions [128].…”

Section: Non-iidness and Data Quality In Fl-based Smart Healthcarementioning

confidence: 99%

Federated Learning for Smart Healthcare: A Survey

Nguyen¹,

Pham²,

Pathirana³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent advances in communication technologies and Internet-of-Medical-Things have transformed smart healthcare enabled by artificial intelligence (AI). Traditionally, AI techniques require centralized data collection and processing that may be infeasible in realistic healthcare scenarios due to the high scalability of modern healthcare networks and growing data privacy concerns. Federated Learning (FL), as an emerging distributed collaborative AI paradigm, is particularly attractive for smart healthcare, by coordinating multiple clients (e.g., hospitals) to perform AI training without sharing raw data. Accordingly, we provide a comprehensive survey on the use of FL in smart healthcare. First, we present the recent advances in FL, the motivations, and the requirements of using FL in smart healthcare. The recent FL designs for smart healthcare are then discussed, ranging from resource-aware FL, secure and privacy-aware FL to incentive FL and personalized FL. Subsequently, we provide a state-of-the-art review on the emerging applications of FL in key healthcare domains, including health data management, remote health monitoring, medical imaging, and COVID-19 detection. Several recent FL-based smart healthcare projects are analyzed, and the key lessons learned from the survey are also highlighted. Finally, we discuss interesting research challenges and possible directions for future FL research in smart healthcare.

show abstract

FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

Cited by 60 publications

References 25 publications

SemiFL: Communication Efficient Semi-Supervised Federated Learning with Unlabeled Clients

SemiFL: Communication Efficient Semi-Supervised Federated Learning with Unlabeled Clients

Memory-aware curriculum federated learning for breast cancer classification

Federated Learning for Smart Healthcare: A Survey

Contact Info

Product

Resources

About