FedCon: A Contrastive Framework for Federated Semi-Supervised Learning

Long, Zewei; Wang, Jiaqi; Wang, Yaqing; Xiao, Houping; Ma, Fenglong

doi:10.48550/arxiv.2109.04533

Cited by 4 publications

(6 citation statements)

References 23 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, two federated self-supervised learning frameworks for images with limited labels was proposed in [39], based on federated CLR with feature sharing (FedCLF). In contrast to previous approaches that assume labeled data are available at the client-side, Long et al [26] introduced FedCon, a novel framework designed to address scenarios where local client data is unlabeled and only the server has access to labeled data. FedCon tackles this challenge by employing a contrastive network architecture, which consists of two subnetworks, enabling effective handling of the unlabeled data at the client-side.…”

Section: Federated Learningmentioning

confidence: 99%

“…Research groups have devoted resources for approaches specifically designed for federated settings, leveraging the power of representation learning while respecting the decentralized nature of data. These approaches often involve a range of learning techniques, such as self-supervised, semi-supervised, unsupervised learning, and transfer learning [44,39,26,18,11]. In the context of federated learning This ICCV workshop paper is the Open Access version, provided by the Computer Vision Foundation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FedLID: Self-Supervised Federated Learning for Leveraging Limited Image Data

Psaltis,

Kastellos,

Patrikakis

et al. 2023

2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

View full text Add to dashboard Cite

This study investigates the challenging task of training visual models with very few available data, further complicated by the distribution being imbalanced and scattered across nodes. To address this diverse availability of training data in different federated settings, a customized selfsupervised learning approach tailored specifically for each scenario is being proposed. In particular, a hybrid approach combining self-supervised and supervised learning techniques under a federated umbrella has been utilized at both the global and local level, harnessing the potential of unlabeled data. Extensive experiments provide a detailed analysis of the problem at hand and demonstrate the particular characteristics of the proposed learning schemes in distributed scenarios. The overall proposed approach achieves superior recognition performance in the currently broadest public dataset, surpassing all baselines by a substantial margin. The proposed solution can operate efficiently at a local level without prior knowledge of the characteristics or distribution of data across nodes.

show abstract

Section: Federated Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FedLID: Self-Supervised Federated Learning for Leveraging Limited Image Data

Psaltis,

Kastellos,

Patrikakis

et al. 2023

2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

View full text Add to dashboard Cite

show abstract

“…This scenario is referred to as FSSL with global semisupervision [40]. In other applications where large labeled datasets are more readily available or can be transferred, the roles of the aggregating server and the labeled data owner may overlap, and this scenario is instead referred to as the labels-at-server scenario [34]- [38].…”

Section: B Federated Learning With Unlabeled Datamentioning

confidence: 99%

“…[37] adapted a combination of two stateof-the-art semi-supervised methods, FixMatch [17] and Mix-Match [19], in a federated setting. [38] and [39] proposed methods based on contrastive learning and knowledge distillation, respectively. Apart from classification, FSSL has been also used for the task of COVID-19 region segmentation in chest computed tomography scans [40].…”

Section: B Federated Learning With Unlabeled Datamentioning

confidence: 99%

Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases

Kassem¹,

Alapatt²,

Mascagni³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent advancements in deep learning methods bring computer-assistance a step closer to fulfilling promises of safer surgical procedures. However, the generalizability of such methods is often dependent on training on diverse datasets from multiple medical institutions, which is a restrictive requirement considering the sensitive nature of medical data. Recently proposed collaborative learning methods such as Federated Learning (FL) allow for training on remote datasets without the need to explicitly share data. Even so, data annotation still represents a bottleneck, particularly in medicine and surgery where clinical expertise is often required. With these constraints in mind, we propose FedCy, a federated semi-supervised learning (FSSL) method that combines FL and self-supervised learning to exploit a decentralized dataset of both labeled and unlabeled videos, thereby improving performance on the task of surgical phase recognition. By leveraging temporal patterns in the labeled data, FedCy helps guide unsupervised training on unlabeled data towards learning taskspecific features for phase recognition. We demonstrate significant performance gains over state-of-the-art FSSL methods on the task of automatic recognition of surgical phases using a newly collected multi-institutional dataset of laparoscopic cholecystectomy videos. Furthermore, we demonstrate that our approach also learns more generalizable features when tested on data from an unseen domain.

show abstract

“…In detail, the authors used two different data augmentation methods to transform the unlabeled data, and supervised the strongly augmented samples with sharpened low-entropy prediction on the weakly augmented samples. Since the semi-supervised learning problem also exists in the FL scenario, some current studies [13], [32] and [33] investigated to construct SSL framework for FL based applications. As an instance, Jeong et al in [13] presented an inter-client consistency loss and a disjoint learning pattern on labeled and unlabeled data.…”

Section: Related Workmentioning

confidence: 99%

Uncertainty Minimization for Personalized Federated Semi-Supervised Learning

Shi¹,

Chen²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

Since federated learning (FL) has been introduced as a decentralized learning technique with privacy preservation, statistical heterogeneity of distributed data stays the main obstacle to achieve robust performance and stable convergence in FL applications. Model personalization methods have been studied to overcome this problem. However, existing approaches are mainly under the prerequisite of fully labeled data, which is unrealistic in practice due to the requirement of expertise. The primary issue caused by partial-labeled condition is that, clients with deficient labeled data can suffer from unfair performance gain because they lack adequate insights of local distribution to customize the global model. To tackle this problem, 1) we propose a novel personalized semi-supervised learning paradigm which allows partial-labeled or unlabeled clients to seek labeling assistance from data-related clients (helper agents), thus to enhance their perception of local data; 2) based on this paradigm, we design an uncertainty-based data-relation metric to ensure that selected helpers can provide trustworthy pseudo labels instead of misleading the local training; 3) to mitigate the network overload introduced by helper searching, we further develop a helper selection protocol to achieve efficient communication with negligible performance sacrifice. Experiments show that our proposed method can obtain superior performance and more stable convergence than other related works with partial labeled data, especially in highly heterogeneous setting.

show abstract

FedCon: A Contrastive Framework for Federated Semi-Supervised Learning

Cited by 4 publications

References 23 publications

FedLID: Self-Supervised Federated Learning for Leveraging Limited Image Data

FedLID: Self-Supervised Federated Learning for Leveraging Limited Image Data

Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases

Uncertainty Minimization for Personalized Federated Semi-Supervised Learning

Contact Info

Product

Resources

About