FedTriNet: A Pseudo Labeling Method with Three Players for Federated Semi-supervised Learning

Che, Liwei; Long, Zewei; Wang, Jiaqi; Wang, Yaqing; Xiao, Houping; Ma, Fenglong

doi:10.1109/bigdata52589.2021.9671374

Cited by 15 publications

(10 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al incorporated graph normalization into FSSL such that the gradient diversity from the different users can be mitigated, thereby improving testing accuracy [69]. Che et al proposed FedTriNet, an FSSL framework that exploits the subtlydesigned labeling mechanism to augment the insufficient amount of labeled data such that the test accuracy can be improved [70]. Kang et al presented FedCVT to improve the testing accuracy when the amount of labeled data is insufficient.…”

Section: Related Workmentioning

confidence: 99%

Dual-Objective Personalized Federated Service System With Partially-Labeled Data Over Wireless Networks

Ching

Chang

Kuo

et al. 2023

IEEE Trans. Serv. Comput.

View full text Add to dashboard Cite

Federated learning (FL) emerges to mitigate the privacy concerns in machine learning-based services and applications, and personalized federated learning (PFL) evolves to alleviate the issue of data heterogeneity. However, FL and PFL usually rest on two assumptions: the users' data is well-labeled, or the personalized goals align with sufficient local data. Unfortunately, the two assumptions may not hold in most cases, where data labeling is costly, or most users have no sufficient local data to satisfy their personalized needs. To this end, we first formulate the problem, DoLP, that studies the issue of insufficient and partially-labeled data on FL-based services. DoLP aims to maximize two service objectives: 1) personalized classification objective and 2) the personalized labeling objective for each user within the constraint of training time over wireless networks. Then, we propose a PFL-based service system DoFed-SPP to solve DoLP. The DoFed-SPP's novelty is two-fold. First, we devise an inference-based first-order approximation metric, similarity ratio, to identify the similarity between users' local data. Second, we design an approximation algorithm to determine the appropriate size and set of users for uploading in each round. Extensive experiments show DoFed-SPP outperforms the state-of-the-art in final accuracy and time-to-accuracy performance on CIFAR10/100 and DBPedia.

show abstract

Section: Related Workmentioning

confidence: 99%

Dual-Objective Personalized Federated Service System With Partially-Labeled Data Over Wireless Networks

Ching

Chang

Kuo

et al. 2023

IEEE Trans. Serv. Comput.

View full text Add to dashboard Cite

show abstract

“…In [1], the authors add self-ensemble learning and complementary negative learning to federated learning. FedTriNet [2] uses three networks and a dynamic quality control mechanism to generate high-quality pseudo labels for unlabeled data, and then the pseudo labeled and real labeled data are used together to retrain the model.…”

Section: Semi-supervised Federated Learningmentioning

confidence: 99%

“…Recently, a few SemiFL approaches have been proposed, such as FedMatch [8], SSFL [39], FedMix [38], FedSEAL [1], SemiFL [4], and FedTriNet [2]. These models primarily focus on developing a good global model and serving one global model to all clients.…”

Section: Introductionmentioning

confidence: 99%

Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT

Wang¹,

Shenglai²,

Long³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning (FL) enables multiple clients to train models collaboratively without sharing local data, which has achieved promising results in different areas, including the Internet of Things (IoT). However, end IoT devices do not have abilities to automatically annotate their collected data, which leads to the label shortage issue at the client side. To collaboratively train an FL model, we can only use a small number of labeled data stored on the server. This is a new yet practical scenario in federated learning, i.e., labels-at-server semi-supervised federated learning (SemiFL). Although several SemiFL approaches have been proposed recently, none of them can focus on the personalization issue in their model design. IoT environments make SemiFL more challenging, as we need to take device computational constraints and communication cost into consideration simultaneously. To tackle these new challenges together, we propose a novel SemiFL framework named pFedKnow. pFedKnow generates lightweight personalized client models via neural network pruning techniques to reduce communication cost. Moreover, it incorporates pretrained large models as prior knowledge to guide the aggregation of personalized client models and further enhance the framework performance. Experiment results on both image and text datasets show that the proposed pFedKnow outperforms state-of-the-art baselines as well as reducing considerable communication cost. The source code of the proposed pFedKnow is available at https://github.com/JackqqWang/pfedknow/ tree/master.

show abstract

“…Other works proposed methods to enhance pseudolabels. [32] utilized a dynamic thresholding strategy for selecting pseudo-labels based on model confidence along with having multiple clients vote to generate pseudo-labels. [33] utilized peer learning and ensemble averaging from multiple clients.…”

Section: B Federated Learning With Unlabeled Datamentioning

confidence: 99%

Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases

Kassem¹,

Alapatt²,

Mascagni³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent advancements in deep learning methods bring computer-assistance a step closer to fulfilling promises of safer surgical procedures. However, the generalizability of such methods is often dependent on training on diverse datasets from multiple medical institutions, which is a restrictive requirement considering the sensitive nature of medical data. Recently proposed collaborative learning methods such as Federated Learning (FL) allow for training on remote datasets without the need to explicitly share data. Even so, data annotation still represents a bottleneck, particularly in medicine and surgery where clinical expertise is often required. With these constraints in mind, we propose FedCy, a federated semi-supervised learning (FSSL) method that combines FL and self-supervised learning to exploit a decentralized dataset of both labeled and unlabeled videos, thereby improving performance on the task of surgical phase recognition. By leveraging temporal patterns in the labeled data, FedCy helps guide unsupervised training on unlabeled data towards learning taskspecific features for phase recognition. We demonstrate significant performance gains over state-of-the-art FSSL methods on the task of automatic recognition of surgical phases using a newly collected multi-institutional dataset of laparoscopic cholecystectomy videos. Furthermore, we demonstrate that our approach also learns more generalizable features when tested on data from an unseen domain.

show abstract

FedTriNet: A Pseudo Labeling Method with Three Players for Federated Semi-supervised Learning

Cited by 15 publications

References 5 publications

Dual-Objective Personalized Federated Service System With Partially-Labeled Data Over Wireless Networks

Dual-Objective Personalized Federated Service System With Partially-Labeled Data Over Wireless Networks

Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT

Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases

Contact Info

Product

Resources

About