FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

Zhang, Bowen; Wang, Yidong; Hou, Wenxin; Wu, Hao; Wang, Jindong; Okumura, Manabu; Shinozaki, Takahiro

doi:10.48550/arxiv.2110.08263

Cited by 5 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also implement three commonly used pseudo-label-based (PL) strategies, in which a pseudo label is assigned to each unlabelled noisy instance. PL-H (Zhang et al 2021)/PL-S (Li, Socher, and Hoi 2019) generates a hard/soft label for unlabelled instances according to model prediction. PL-K (Ortego et al 2021) utilizes k-NN on the reference set Ω to predict pseudo label for instances in D û.…”

Section: Regularization On Unlabelled Noisy Samplementioning

confidence: 99%

Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videos

Wang

Weng

Yuan

et al. 2023

AAAI

View full text Add to dashboard Cite

Learning with noisy label is a classic problem that has been extensively studied for image tasks, but much less for video in the literature. A straightforward migration from images to videos without considering temporal semantics and computational cost is not a sound choice. In this paper, we propose two new strategies for video analysis with noisy labels: 1) a lightweight channel selection method dubbed as Channel Truncation for feature-based label noise detection. This method selects the most discriminative channels to split clean and noisy instances in each category. 2) A novel contrastive strategy dubbed as Noise Contrastive Learning, which constructs the relationship between clean and noisy instances to regularize model training. Experiments on three well-known benchmark datasets for video classification show that our proposed truNcatE-split-contrAsT (NEAT) significantly outperforms the existing baselines. By reducing the dimension to 10% of it, our method achieves over 0.4 noise detection F1-score and 5% classification accuracy improvement on Mini-Kinetics dataset under severe noise (symmetric-80%). Thanks to Noise Contrastive Learning, the average classification accuracy improvement on Mini-Kinetics and Sth-Sth-V1 is over 1.6%.

show abstract

Section: Regularization On Unlabelled Noisy Samplementioning

confidence: 99%

Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videos

Wang

Weng

Yuan

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…In semi-supervised learning in image classification, there were some approaches, such as Dash [68] and Flex-Match [69], to measure the uncertainty of pseudo-labels by applying a different threshold to each sample, unlike FixMatch [57] which assumes that the samples exceed the hand-crafted confidence threshold. In contrast to the existing semi-supervised method [57,68,69] which assumes that all pseudo-labels are certain without considering the uncertainty of the generated pseudo-label, we improve the performance by considering the uncertainty of the pseudo-label.…”

Section: Related Workmentioning

confidence: 99%

Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels

Kim¹,

Ryoo²,

Seo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Establishing dense correspondences across semantically similar images remains a challenging task due to the significant intra-class variations and background clutters. Traditionally, a supervised learning was used for training the models, which required tremendous manually-labeled data, while some methods suggested a self-supervised or weaklysupervised learning to mitigate the reliance on the labeled data, but with limited performance.In this paper, we present a simple, but effective solution for semantic correspondence that learns the networks in a semi-supervised manner by supplementing few ground-truth correspondences via utilization of a large amount of confident correspondences as pseudo-labels, called SemiMatch. Specifically, our framework generates the pseudo-labels using the model's prediction itself between source and weakly-augmented target, and uses pseudo-labels to learn the model again between source and strongly-augmented target, which improves the robustness of the model. We also present a novel confidence measure for pseudo-labels and data augmentation tailored for semantic correspondence. In experiments, SemiMatch achieves state-of-the-art performance on various benchmarks, especially on PF-Willow by a large margin.* Equal contribution † Corresponding author (a) Supervised loss (b) Self-supervised loss (c) Unsupervised loss (Ours) (d) Semi-supervised loss (Ours)

show abstract

“…Pseudo-label methods generate artificial labels for some unlabeled images and then train the model with these artificial labels, while consistency regularization tries to obtain an artificial distribution/label and applied it as a supervision signal with other augmentations/views. These two strategies have been adopted by a number of recent SSL works [4,8,[29][30][31][32][33][34][35][36][37][38][39]. For example, FixMatch [4] proposes a simple combination of pseudo labels and consistency regularization.…”

Section: Semi-supervised Learningmentioning

confidence: 99%

DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples

Ding

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

The scarcity of labeled data is a critical obstacle to deep learning. Semi-supervised learning (SSL) provides a promising way to leverage unlabeled data by pseudo labels. However, when the size of labeled data is very small (say a few labeled samples per class), SSL performs poorly and unstably, possibly due to the low quality of learned pseudo labels. In this paper, we propose a new SSL method called DP-SSL that adopts an innovative data programming (DP) scheme to generate probabilistic labels for unlabeled data. Different from existing DP methods that rely on human experts to provide initial labeling functions (LFs), we develop a multiple-choice learning (MCL) based approach to automatically generate LFs from scratch in SSL style. With the noisy labels produced by the LFs, we design a label model to resolve the conflict and overlap among the noisy labels, and finally infer probabilistic labels for unlabeled samples. Extensive experiments on four standard SSL benchmarks show that DP-SSL can provide reliable labels for unlabeled data and achieve better classification performance on test sets than existing SSL methods, especially when only a small number of labeled samples are available. Concretely, for CIFAR-10 with only 40 labeled samples, DP-SSL achieves 93.82% annotation accuracy on unlabeled data and 93.46% classification accuracy on test data, which are higher than the SOTA results.

show abstract

FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

Cited by 5 publications

References 21 publications

Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videos

Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videos

Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels

DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples

Contact Info

Product

Resources

About