ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning

Abstract

The state of the art in semantic segmentation is steadily increasing in performance, resulting in more precise and reliable segmentations in many different applications. However, progress is limited by the cost of generating labels for training, which sometimes requires hours of manual labor for a single image. Because of this, semi-supervised methods have been applied to this task, with varying degrees of success. A key challenge is that common augmentations used in semi-supervised classification are less effective for semantic segmentation. We propose a novel data augmentation mechanism called ClassMix, which generates augmentations by mixing unlabelled samples, by leveraging on the network's predictions for respecting object boundaries. We evaluate this augmentation technique on two common semi-supervised semantic segmentation benchmarks, showing that it attains state-of-theart results. Lastly, we also provide extensive ablation studies comparing different design decisions and training regimes. Code is available at: https://github.com/WilhelmT/ClassMix.

Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation

“…19 urban-scene semantic classes are defined in Cityscapes for semantic segmentation. Similar to previous standards [26,42,19,44,18,41] in semi-supervised semantic segmentation, we randomly sample 1/8 and 1/4 training images to construct the labeled set, and the remaining training images consist of the unlabeled set. To further explore the effectiveness of the proposed method, we also conduct experiments on the PASCAL VOC 2012 dataset (VOC12) [15], which provides 20 semantic classes and 1 background class.…”

Every Annotation Counts: Multi-label Deep Supervision for Medical Image Segmentation

“…Recently, many attempts try to regularize the learner to form predictions invariant towards perturbations [34,45]. Along this line of thought, the data augmentation strategy Cut-Mix [65] can serve as perturbation by mixing predictions and assembling new informative pseudo-labels [19,44]. This approach is often accompanied by mean-teachers [58] serving as robust models for distilling invariance into the student.…”

“…Mean-teachers (i.e. the exponential moving average over previous model parameters) were successfully introduced for semi-supervised classification [58] and saw some use in segmentation [19,44,48]. The idea is that better predictions can be obtained by maintaining a teacher model via continuously updating its parameters with the moving average of the student model and the previous teacher model.…”

“…We extend this by leveraging the meanteacher to provide pseudo-labels for our Multi-label Deep Supervision that the student network is trained with. Previous adaptations to semantic segmentation either did not perturb the input image to the teacher [48] or used Cut-Mix variants [19,44] as perturbations. Instead, we propose to perturb the input image to the teacher in a different way as the student input and still obtain matching outputs.…”

Mask-based Data Augmentation for Semi-supervised Semantic Segmentation

“…While consistency regularization has been successfully employed for classification tasks, applying traditional data augmentation techniques to semantic segmentation has been shown [5] to be less effective as semantic segmentation may not exhibit low density regions around class boundaries. Several approaches have been developed to address this issue by applying augmentation on encoded space instead of input space [6], or by enforcing consistent predictions for unsupervised mixed samples as in CutMix [7,5], CowMix [8], and ClassMix [9].…”