Unsupervised Domain Adaptation in Semantic Segmentation: a Review

Toldo, Marco; Maracani, Andrea; Michieli, Umberto; Zanuttigh, Pietro

doi:10.48550/arxiv.2005.10876

Cited by 13 publications

(24 citation statements)

References 104 publications

(220 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since our work is built upon the mean teacher framework, we briefly review related methods in this section. For other UDA methods for semantic segmentation, such as style transfer and adversarial learning, we recommend the excellent survey [32]. Mean teacher is a widely used framework in the field of semi-supervised learning, which is based on the simple idea that under the supervision of labeled data, unlabeled data should produce consistent predictions under different perturbations.…”

Section: Related Work 21 Mean Teacher-based Methodsmentioning

confidence: 99%

DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation

Gao

Zhang

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Unsupervised domain adaptation (UDA) for semantic segmentation aims to adapt a segmentation model trained on the labeled source domain to the unlabeled target domain. Existing methods try to learn domain invariant features while suffering from large domain gaps that make it difficult to correctly align discrepant features, especially in the initial training phase. To address this issue, we propose a novel Dual Soft-Paste (DSP) method in this paper. Specifically, DSP selects some classes from a source domain image using a long-tail class first sampling strategy and softly pastes the corresponding image patch on both the source and target training images with a fusion weight. Technically, we adopt the mean teacher framework for domain adaptation, where the pasted source and target images go through the student network while the original target image goes through the teacher network. Output-level alignment is carried out by aligning the probability maps of the target fused image from both networks using a weighted crossentropy loss. In addition, feature-level alignment is carried out by aligning the feature maps of the source and target images from student network using a weighted maximum mean discrepancy loss. DSP facilitates the model learning domain-invariant features from the intermediate domains, leading to faster convergence and better performance. Experiments on two challenging benchmarks demonstrate the superiority of DSP over state-of-the-art methods.Code is available at https://github.com/GaoLii/DSP. CCS CONCEPTS• Computing methodologies → Computer vision problems.

show abstract

Section: Related Work 21 Mean Teacher-based Methodsmentioning

confidence: 99%

DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation

Gao

Zhang

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…Recently, auxiliary tasks, such as the adaptation of a well-trained model from a similar domain with a similar task [11], [41], have been leveraged to migrate this problem. Although we will not cover the unsupervised segmentation and their solutions, such as unsupervised domain adaptation (UDA) [42] and zero-shot learning [43], we mention it here to start by looking at all settings in the big picture. In this paper, we focus on methods that learn to segment medical images with incomplete, inexact, and inaccurate annotations by jointly leveraging a few labeled data and a large number of unlabeled examples.…”

Section: Overviewmentioning

confidence: 99%

“…Thus, pretraining on relevant domains and applying to the current domain with supervised or semi-supervised training, known as Domain Adaptation [219] or Domain generalization, has received growing attention. Please refer to [42] for comprehensive reviews of domain adaptation for semantic segmentation.…”

Section: K Summarymentioning

confidence: 99%

Medical Image Segmentation With Limited Supervision: A Review of Deep Network Models

Wang

2021

IEEE Access

View full text Add to dashboard Cite

Despite the remarkable performance of deep learning methods on various tasks, most cutting-edge models rely heavily on large-scale annotated training examples, which are often unavailable for clinical and health care tasks. The labeling costs for medical images are very high, especially in medical image segmentation, which typically requires intensive pixel/voxel-wise labeling. Therefore, the strong capability of learning and generalizing from limited supervision, including a limited amount of annotations, sparse annotations, and inaccurate annotations, is crucial for the successful application of deep learning models in medical image segmentation. However, due to its intrinsic difficulty, segmentation with limited supervision is challenging and specific model design and/or learning strategies are needed. In this paper, we provide a systematic and up-to-date review of the solutions above, with summaries and comments about the methodologies. We also highlight several problems in this field, discussed future directions observing further investigations.

show abstract

“…However, such global feature alignment does not necessarily result in the intraclassly correct semantic representation of the target domain. 29 In multiple class segmentation, it almost fails to achieve substantial gains over the baseline that is trained with data augmentation and registration. 42 For some small-gap UDA tasks of medical image segmentation, the generative-based approaches are wildly used.…”

Section: Introductionmentioning

confidence: 99%

IAS‐NET: Joint intraclassly adaptive GAN and segmentation network for unsupervised cross‐domain in neonatal brain MRI segmentation

You

Wang

et al. 2021

Medical Physics

View full text Add to dashboard Cite

In neonatal brain magnetic resonance image (MRI) segmentation, the model we trained on the training set (source domain) often performs poorly in clinical practice (target domain). As the label of target-domain images is unavailable, this cross-domain segmentation needs unsupervised domain adaptation (UDA) to make the model adapt to the target domain. However, the shape and intensity distribution of neonatal brain MRI images across the domains are largely different from adults'. Current UDA methods aim to make synthesized images similar to the target domain as a whole. But it is impossible to synthesize images with intraclass similarity because of the regional misalignment caused by the cross-domain difference. This will result in generating intraclassly incorrect intensity information from target-domain images. To address this issue, we propose an IAS-NET (joint intraclassly adaptive generative adversarial network (GAN) (IA-NET) and segmentation) framework to bridge the gap between the two domains for intraclass alignment. Methods: Our proposed IAS-NET is an elegant learning framework that transfers the appearance of images across the domains from both image and feature perspectives. It consists of the proposed IA-NET and a segmentation network (S-NET). The proposed IA-NET is a GAN-based adaptive network that contains one generator (including two encoders and one shared decoder) and four discriminators for cross-domain transfer. The two encoders are implemented to extract original image, mean, and variance features from source and target domains. The proposed local adaptive instance normalization algorithm is used to perform intraclass feature alignment to the target domain in the feature-map level. S-NET is a U-net structure network that is used to provide semantic constraint by a segmentation loss for the training of IA-NET. Meanwhile, it offers pseudo-label images for calculating intraclass features of the target domain. Source code (in Tensorflow) is available at https://github.com/lb-whu/RAS-NET/. Results: Extensive experiments are carried out on two different data sets (Neo-BrainS12 and dHCP), respectively. There exist great differences in the shape, size, and intensity distribution of magnetic resonance (MR) images in the two databases. Compared to baseline, we improve the average dice score of all tissues on NeoBrains12 by 6% through adaptive training with unlabeled dHCP images. Besides, we also conduct experiments on dHCP and improved the average dice score by 4%. The quantitative analysis of the mean and variance of the synthesized images shows that the synthesized image by the proposed is closer to the target domain both in the full brain or within each class than that of the compared methods.

show abstract

Unsupervised Domain Adaptation in Semantic Segmentation: a Review

Cited by 13 publications

References 104 publications

DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation

DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation

Medical Image Segmentation With Limited Supervision: A Review of Deep Network Models

IAS‐NET: Joint intraclassly adaptive GAN and segmentation network for unsupervised cross‐domain in neonatal brain MRI segmentation

Contact Info

Product

Resources

About