Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation

Marsden, Robert; Bartler, Alexander; Döbler, Mario; Yang, Bin

doi:10.1109/ijcnn55064.2022.9892322

Cited by 23 publications

(14 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unsupervised Domain Adaptation: Methods of unsupervised domain adaptation try to overcome the delta between the training data domain, the so-called source domain, and the target domain, which is unlabeled but relevant. These methods can be categorized into three types: Distribution alignment of the source and target domain data in the input (e.g., Hoffman et al, 6 Yunsheng et al 7 and Yang et al 8 ), feature (e.g., Niemeijer et al, 9 Hoffman et al 10 and Marsden et al 11 ) and, output space (e.g., Vu et al, 12 Tsai et al 13 and Zheng et al 14 ). Distribution alignment in the input space often relies on CycleGANs, which compute a style transformation from the source to the target images domain.…”

Section: Related Workmentioning

confidence: 99%

Overcoming the sensor delta for semantic segmentation in OCT images

Niemeijer¹,

Ehrhardt²,

Kepp³

et al. 2023

Medical Imaging 2023: Computer-Aided Diagnosis

View full text Add to dashboard Cite

The performance of a segmentation network optimized on data from a specific type of OCT sensor will decrease when applied to data from a different sensor. In this work, we deal with the research question of adapting models to data from an unlabeled new sensor with new properties in an unsupervised way. This challenge is known as unsupervised domain adaptation and can alleviate the need for costly manual annotation by radiologists. We show that one can strongly improve a model's result that was trained in a supervised way on the source OCT sensor domain on the target sensor domain. We do this by aligning the source and target domain distributions in the feature space through a semantic clustering method. Apart from the unsupervised domain adaptation, we improved even the supervised training compared to the results in the RETOUCH challenge by employing a sophisticated training strategy. The RETOUCH challenge contains three different types of OCT scanners and provides annotations for the task of disease-related fluid classes.

show abstract

Section: Related Workmentioning

confidence: 99%

Overcoming the sensor delta for semantic segmentation in OCT images

Niemeijer¹,

Ehrhardt²,

Kepp³

et al. 2023

Medical Imaging 2023: Computer-Aided Diagnosis

View full text Add to dashboard Cite

show abstract

“…[18,21,43] employ partially dense contrast between classes using pixel features as anchors and class-level prototype vectors as positives and negatives. Along similar lines, [22,43] implement partially dense contrast between classes using pixel features as anchors and estimated classlevel distributions as positives and negatives, while [28] use class prototypes both as anchors and as positives/negatives. These approaches are prone to false positive/negative samples which contaminate the contrastive loss due to potential errors in the target-domain pseudo-labels, which are used both to determine the anchors and to compute the class prototypes that serve as positives and negatives.…”

Section: Related Workmentioning

confidence: 99%

Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation

Bruggemann¹,

Sakaridis²,

Tim³

et al. 2023

Preprint

View full text Add to dashboard Cite

Standard unsupervised domain adaptation methods adapt models from a source to a target domain using labeled source data and unlabeled target data jointly. In model adaptation, on the other hand, access to the labeled source data is prohibited, i.e., only the source-trained model and unlabeled target data are available. We investigate normal-to-adverse condition model adaptation for semantic segmentation, whereby image-level correspondences are available in the target domain. The target set consists of unlabeled pairs of adverse-and normalcondition street images taken at GPS-matched locations. Our method-CMA-leverages such image pairs to learn condition-invariant features via contrastive learning. In particular, CMA encourages features in the embedding space to be grouped according to their condition-invariant semantic content and not according to the condition under which respective inputs are captured. To obtain accurate cross-domain semantic correspondences, we warp the normal image to the viewpoint of the adverse image and leverage warp-confidence scores to create robust, aggregated features. With this approach, we achieve state-of-the-art semantic segmentation performance for model adaptation on several normal-to-adverse adaptation benchmarks, such as ACDC and Dark Zurich. We also evaluate CMA on a newly procured adverse-condition generalization benchmark and report favorable results compared to standard unsupervised domain adaptation methods, despite the comparative handicap of CMA due to source data inaccessibility. Code is available at https://github.com/brdav/cma.

show abstract

“…This is especially true for the synthetic to real world domain change. Generally speaking, UDA methods aim to align the distributions in the input (e.g., Hoffman et al [7], Yunsheng et al [12], Termöhlen et al [27] and Yang et al [30]), the feature (e.g., Niemeijer et al [17,16], Hoffman et al [8] and Marsden et al [13]), or the output space (e.g., Vu et al [29], Tsai et al [28] and Zheng et al [31]) of a neural network. The recent state of the art is dominated by transformer based architectures that utilize adaptation techniques in the feature and output space as, for instance, presented in [9].…”

Section: Unsupervised Domain Adaptation (Uda)mentioning

confidence: 99%

Synthetic Dataset Acquisition for a Specific Target Domain

Niemeijer,

Mittal,

Brox

2023

2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

View full text Add to dashboard Cite

Intelligent sampling from simulation becomes crucial due to storage and hardware constraints. This research focuses on developing an intelligent acquisition strategy for synthetic data and evaluates multiple approaches to address the limitations of existing domain adaptation methods. Selecting suitable synthetic data for real-world model training presents challenges, as accurately representing the real world remains elusive. We tackle the task of adapting from synthetic to real-world data through unsupervised domain adaptation, a challenging setting for perception systems. The performance of our acquisition function is measured by its facilitation of this adaptation.We showcase different strategies, to assign value to synthetic images. Acquisition functions either operate based on synthetic data alone or take the given real world target domain into account, to assign a value to synthetic images. Leveraging assumptions from semi-supervised learning, we identify challenging real-world images and find their counterparts in the synthetic world. Evaluation is conducted using the GTA-5 dataset as the representative synthetic world and the Cityscapes and ACDC dataset as the target domain. State-of-the-art unsupervised domain adaptation approaches are employed to assess the effectiveness of our acquisition function.By advancing the utilization of synthetic data in training perception systems, this research contributes to improved real-world performance. Our findings demonstrate the potential of intelligent acquisition strategies for enhancing the adaptation from synthetic to real-world domains.

show abstract

Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation

Cited by 23 publications

References 30 publications

Overcoming the sensor delta for semantic segmentation in OCT images

Overcoming the sensor delta for semantic segmentation in OCT images

Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation

Synthetic Dataset Acquisition for a Specific Target Domain

Contact Info

Product

Resources

About