SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learning

Kim, Byoungjip; Choo, Jinho; Kwon, Yeong-Dae; Joe, Seongho; Min, Seung-Jai; Gwon, Youngjune

doi:10.48550/arxiv.2101.06480

Cited by 9 publications

(18 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baseline methods We compare to a number of recent methods such as FixMatch (Sohn et al, 2020), Mix-Match (Berthelot et al, 2019b), DASH (Xu et al, 2021), SelfMatch (Kim et al, 2021), Mean Teacher (Tarvainen and Valpola, 2017), Virtual Adversarial Training (Miyato et al, 2018), and Mixup (Berthelot et al, 2019b).…”

Section: Semi-supervised Learningmentioning

confidence: 99%

Deep Reference Priors: What is the best way to pretrain a model?

Gao¹,

Ramesh²,

Chaudhari³

2022

Preprint

View full text Add to dashboard Cite

What is the best way to exploit extra data-be it unlabeled data from the same task, or labeled data from a related task-to learn a given task? This paper formalizes the question using the theory of reference priors. Reference priors are objective, uninformative Bayesian priors that maximize the mutual information between the task and the weights of the model. Such priors enable the task to maximally affect the Bayesian posterior, e.g., reference priors depend upon the number of samples available for learning the task and for very small sample sizes, the prior puts more probability mass on low-complexity models in the hypothesis space. This paper presents the first demonstration of reference priors for mediumscale deep networks and image-based data. We develop generalizations of reference priors and demonstrate applications to two problems. First, by using unlabeled data to compute the reference prior, we develop new Bayesian semi-supervised learning methods that remain effective even with very few samples per class. Second, by using labeled data from the source task to compute the reference prior, we develop a new pretraining method for transfer learning that allows data from the target task to maximally affect the Bayesian posterior. Empirical validation of these methods is conducted on image classification datasets.

show abstract

Section: Semi-supervised Learningmentioning

confidence: 99%

Deep Reference Priors: What is the best way to pretrain a model?

Gao¹,

Ramesh²,

Chaudhari³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…S 4 L (Zhai et al 2019) integrated two pretext-based self-supervised approaches in SSL and showed that unsupervised representation learning complements existing SSL methods. SelfMatch (Kim et al 2021) pre-trained the model on unlabeled data with SOTA selfsupervised contrastive learning techniques and re-trained on the whole dataset with SSL approaches. In SIMPLE (Hu et al 2021), a revised pair-loss was introduced to explore the relations among unlabeled samples.…”

Section: Related Workmentioning

confidence: 99%

“…All the related works are sorted by their publication date. Results with * was reported in FixMatch(Sohn et al 2020), while results with † comes from the most recent papers(Kim et al 2021;Li, Xiong, and Hoi 2020;Xu et al 2021;Abuduweili et al 2021), respectively.…”

mentioning

confidence: 97%

LaSSL: Label-Guided Self-Training for Semi-supervised Learning

Zhao

Zhou

Wang

et al. 2022

AAAI

View full text Add to dashboard Cite

The key to semi-supervised learning (SSL) is to explore adequate information to leverage the unlabeled data. Current dominant approaches aim to generate pseudo-labels on weakly augmented instances and train models on their corresponding strongly augmented variants with high-confidence results. However, such methods are limited in excluding samples with low-confidence pseudo-labels and under-utilization of the label information. In this paper, we emphasize the cruciality of the label information and propose a Label-guided Self-training approach to Semi-supervised Learning (LaSSL), which improves pseudo-label generations from two mutually boosted strategies. First, with the ground-truth labels and iteratively-polished pseudo-labels, we explore instance relations among all samples and then minimize a class-aware contrastive loss to learn discriminative feature representations that make same-class samples gathered and different-class samples scattered. Second, on top of improved feature representations, we propagate the label information to the unlabeled samples across the potential data manifold at the feature-embedding level, which can further improve the labelling of samples with reference to their neighbours. These two strategies are seamlessly integrated and mutually promoted across the whole training process. We evaluate LaSSL on several classification benchmarks under partially labeled settings and demonstrate its superiority over the state-of-the-art approaches.

show abstract

“…To maximize the value of the limited labels, existing works either try to maintain the consistency by competing for the introduced perturbations [45], [46] or seek the relationship among different samples [47], [48]. Self-supervised learning [49]- [52] is a feasible way to learn the visual representation for semi-supervised learning, which can somehow be a complement to the lack of annotations. Specific to medical image segmentation, Xia et al [53] proposed uncertainty-aware multi-view co-training for 3D volumetric medical image segmentation.…”

Section: B Reducing Annotation Efforts For Medical Image Segmentationmentioning

confidence: 99%

Multi-Layer Pseudo-Supervision for Histopathology Tissue Semantic Segmentation using Patch-level Classification Labels

Han¹,

Lin²,

Mai³

et al. 2021

Preprint

View full text Add to dashboard Cite

Tissue-level semantic segmentation is a vital step in computational pathology. Fully-supervised models have already achieved outstanding performance with dense pixel-level annotations. However, drawing such labels on the giga-pixel whole slide images is extremely expensive and time-consuming. In this paper, we use only patch-level classification labels to achieve tissue semantic segmentation on histopathology images, finally reducing the annotation efforts. We proposed a two-step model including a classification and a segmentation phases. In the classification phase, we proposed a CAM-based model to generate pseudo masks by patch-level labels. In the segmentation phase, we achieved tissue semantic segmentation by our proposed Multi-Layer Pseudo-Supervision. Several technical novelties have been proposed to reduce the information gap between pixellevel and patch-level annotations. As a part of this paper, we introduced a new weakly-supervised semantic segmentation (WSSS) dataset for lung adenocarcinoma (LUAD-HistoSeg). We conducted several experiments to evaluate our proposed model on two datasets. Our proposed model outperforms two state-ofthe-art WSSS approaches. Note that we can achieve comparable quantitative and qualitative results with the fully-supervised model, with only around a 2% gap for MIoU and FwIoU. By comparing with manual labeling, our model can greatly save

show abstract

SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learning

Cited by 9 publications

References 12 publications

Deep Reference Priors: What is the best way to pretrain a model?

Deep Reference Priors: What is the best way to pretrain a model?

LaSSL: Label-Guided Self-Training for Semi-supervised Learning

Multi-Layer Pseudo-Supervision for Histopathology Tissue Semantic Segmentation using Patch-level Classification Labels

Contact Info

Product

Resources

About