Semi-Supervised Learning with Scarce Annotations

ReLaB: Reliable Label Bootstrapping for Semi-Supervised Learning

“…Recent contributions shows that coupling self-supervised and semi-supervised learning can increase the accuracy when few labels are available. Rebuffi et al [27] use RotNet [23] as a network initialization strategy, ReMixMatch [18] exploits RotNet [23] together with their semi-supervised algorithm to achieve stability with few labels, and EnAET [26] leverage transformation encoding from AET [39] to improve the consistency of predictions on transformed images.…”

“…We combine our approach with state-ofthe-art pseudo-labeling [17] and consistency regularizationbased [18] semi-supervised methods to prove the stability of ReLaB when applied to different semi-supervised strategies. We use the default configuration for pseudo-labeling 1 except for the network initialization, where we make use of the Rotation self-supervised objective [23] and freeze all the layers up to the last convolutional block in a similar fashion to Rebufi et al [27]. We find that this is necessary to preserve strong early features throughout the training.…”

“…The success of self-supervised learning has motivated its adoption for semi-supervised learning, which improved performance in cases of very low label availability [18], [26]. Berthelot et al [18] and Wang et al [26] use self-supervision as a regularization which stabilizes network training, while Rebuffi et al [27] make use of selfsupervision [23] as an initialization strategy for semi-supervised training. In this paper, we explore the idea of automatically annotating image data using label propagation.…”

End-to-end novel visual categories learning via auxiliary self-supervision

“…Such self-supervision signals apply to both labeled and unlabeled data, and are argued to be able to teach models on studying high-level features. Instead of utilizing self-supervision as a pretraining stage, which is commonly adopted by most semi-supervised learning algorithms [16,14,19], self-supervision signals are incorporated with pairwise similarity information to supervise the model simultaneously in our proposed algorithm. Thus, supervising signals injected to the model will have strong regularization so that performance degradation caused by noisy and biased pairwise pseudo labels can be largely alleviated.…”

Deep feature learning for image classification via countering over-fitting

“…Such self-supervision signals apply to both labeled and unlabeled data, and are argued to be able to teach models on studying high-level features. Instead of utilizing self-supervision as a pre-training stage, which is commonly adopted for semi-supervised learning purpose [20], [81], [134], self-supervision signals are incorporated with pairwise similarity information to supervise the model simultaneously in the proposed algorithm. Thus, supervising signals injected to the model will have strong regularization so that performance degradation caused by noisy and biased pairwise pseudo labels can be largely alleviated.…”