ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning

Olsson, Viktor; Tranheden, Wilhelm; Pinto, Juliano; Svensson, Lennart

doi:10.1109/wacv48630.2021.00141

Cited by 276 publications

(177 citation statements)

References 19 publications

Supporting

Mentioning

175

Contrasting

Order By: Relevance

“…To address both problems, instead of storing whole images from the previous tasks {1, ..., t − 1}, we propose to store an informative portion that we will mix with the images of the current task t. Image mixing is popular for classification [93], [94], [95], [96], [97], [98] yet, to the best of our knowledge, sees limited use for semantic segmentation [99], [100], [101], [102], [103], and has never been considered to design memory-efficient rehearsal learning systems. Formally, given an image I and the corresponding ground truth segmentation maps S t , we define a binary mask Π c such that ∀c ∈ C t : with O c the selected object for class c. By nature, this patch is extremely sparse and can be efficiently stored on disk by modern compression algorithms [104].…”

Section: Object Rehearsalmentioning

confidence: 99%

Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation

Douillard,

Chen,

Dapogny

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep learning approaches are nowadays ubiquitously used to tackle computer vision tasks such as semantic segmentation, requiring large datasets and substantial computational power. Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes. However, continual learning methods are usually prone to catastrophic forgetting. This issue is further aggravated in CSS where, at each step, old classes from previous iterations are collapsed into the background. In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long-and short-range spatial relationships at feature level. Furthermore, we design an entropy-based pseudo-labelling of the background w.r.t. classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes. Finally, we introduce a novel rehearsal method that is particularly suited for segmentation. Our approach, called PLOP, significantly outperforms state-of-the-art methods in existing CSS scenarios, as well as in newly proposed challenging benchmarks.

show abstract

Section: Object Rehearsalmentioning

confidence: 99%

Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation

Douillard,

Chen,

Dapogny

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…French et al [13] find that mask-based augmentation strategies are effective and introduce an adapted version of a popular technique CutMix [13]. The idea of CutMix is to mix samples by replacing the image region with a patch from another image, which can be regarded as an extension of Cutout [35] and Mixup [36], and is further extended in recent works [14,37,38]. Our approach also adopts the idea from CutMix [13] to enforce consistency between the mixed outputs and the prediction over the mixed inputs.…”

Section: Related Workmentioning

confidence: 99%

“…We comprehensively compare our method with state-of-the-art methods which can be categories as: 1) adversarial based methods, i.e., AdvSSL [58] and S4GAN [66]; 2) methods that encourage consistency, i.e., ICT [6], CutMix [13], CowMix [59] and ClassMix [14]; 3) two recent works that also collaboratively train networks, i.e., ECS [23] and DMT [19]. For fair comparisons, we report the performance reported by their original papers under the same SSL setting.…”

Section: Comparisons With Previous Workmentioning

confidence: 99%

“…Recent state-ofthe-art methods, e.g., MeanTeacher [4], FixMatch [7] and Noisy student [8], share a similar philosophy and enjoy the merit of both pseudo labeling [9,8,10] and consistency regularization [2,3,4,1,11]: the teacher model generates pseudo labels for weakly-augmented unlabeled data whereas the student model trains on the strong-augmented counterparts, as shown in Figure 1a. Such strategy can also be applied to semantic segmentation, and the resulting approaches [12,13,14,15,16,17,18,19] have demonstrated outstanding performance. However, pseudo labeling methods are plagued with confirmation bias [20], i.e., the student model is prone to overfit the erroneous pseudo labels.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robust Adversarial Learning For Semi-Supervised Semantic Segmentation

Zhang

et al. 2020

2020 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Recent semi-supervised learning (SSL) methods are commonly based on pseudo labeling. Since the SSL performance is greatly influenced by the quality of pseudo labels, mutual learning has been proposed to effectively suppress the noises in the pseudo supervision. In this work, we propose robust mutual learning that improves the prior approach in two aspects. First, the vanilla mutual learners suffer from the coupling issue that models may converge to learn homogeneous knowledge. We resolve this issue by introducing mean teachers to generate mutual supervisions so that there is no direct interaction between the two students. We also show that strong data augmentations, model noises and heterogeneous network architectures are essential to alleviate the model coupling. Second, we notice that mutual learning fails to leverage the network's own ability for pseudo label refinement. Therefore, we introduce self-rectification that leverages the internal knowledge and explicitly rectifies the pseudo labels before the mutual teaching. Such self-rectification and mutual teaching collaboratively improve the pseudo label accuracy throughout the learning. The proposed robust mutual learning demonstrates state-of-the-art performance on semantic segmentation in low-data regime.

show abstract

“…A common practice when dealing with domain shift is to annotate some data from the domain of interest and re-train (fine-tune) the network on this new data. Additionally, Semi-Supervised Learning (SSL) methods for semantic segmentation have been proposed [4,5,6,7,8,9], effectively training on a small amount of labelled data by relying on complementary learning from unlabelled data. These approaches are not always feasible as sometimes no annotations at all are accessible in the target domain.…”

Section: Introductionmentioning

confidence: 99%

DACS: Domain Adaptation via Cross-domain Mixed Sampling

Tranheden¹,

Olsson²,

Pinto³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Semantic segmentation models based on convolutional neural networks have recently displayed remarkable performance for a multitude of applications. However, these models typically do not generalize well when applied on new domains, especially when going from synthetic to real data. Unsupervised domain adaptation (UDA) attempts to train on labelled data from one domain (source domain), and simultaneously learn from unlabelled data in the domain of interest (target domain). Existing methods have seen success by training on pseudo-labels for these unlabelled images. Multiple techniques have been proposed to mitigate low-quality pseudo-labels arising from the domain shift, with varying degrees of success. We propose DACS: Domain Adaptation via Cross-domain mixed Sampling, which mixes images from the two domains along with the corresponding labels. These mixed samples are then trained on, in addition to the labelled data itself. We demonstrate the effectiveness of our solution by achieving state-of-the-art results for two common synthetic-toreal semantic segmentation benchmarks for UDA. Code is available at: https://github.com/ vikolss/DACS.

show abstract

ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning

Cited by 276 publications

References 19 publications

Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation

Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation

Robust Adversarial Learning For Semi-Supervised Semantic Segmentation

DACS: Domain Adaptation via Cross-domain Mixed Sampling

Contact Info

Product

Resources

About