Domain Adaptive Semantic Segmentation via Regional Contrastive Consistency Regularization

Abstract

Unsupervised domain adaptation (UDA) aims to bridge the domain shift between the labeled source domain and the unlabeled target domain. However, most existing works perform the global-level feature alignment for semantic segmentation, while the local consistency between the regions has been largely neglected, and these methods are less robust to changing of outdoor environments. Motivated by the above facts, we propose a novel and fully end-to-end trainable approach, called regional contrastive consistency regularization (RCCR) for domain adaptive semantic segmentation. Our core idea is to pull the similar regional features extracted from the same location of different images to be closer, and meanwhile push the features from the different locations of the two images to be separated. We innovatively propose momentum projector heads, where the teacher projector is the exponential moving average of the student. Besides, we present a region-wise contrastive loss with two sampling strategies to realize effective regional consistency. Finally, a memory bank mechanism is designed to learn more robust and stable region-wise features under varying environments. Extensive experiments on two common UDA benchmarks, i.e., GTAV to Cityscapes and SYN-THIA to Cityscapes, demonstrate that our approach outperforms the state-of-the-art methods.

Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

“…PLCA [113] adopts pixel-level cycle association in conducting positive pairs. Zhou et al [114] apply regional contrastive consistency regularization. T2S-DA [29] introduces an image translation engine to ensure cross-domain positive pairs are matched precisely.…”

Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving

“…The style transfer is the most popular approach and is usually performed utilizing feature transforms [60], [61], [63], GAN-based networks, e.g., CycleGANs [64], normalization techniques, such as AdaIN [65], histogram matching [66], [67], [68], or image processing in the frequency domain [69], [70]. In recent years several image content mixing methods were proposed that mix the source and target directly in the image domain [71], [72], [73], [74], [75], [76], [77], [78]. Also, several data augmentation methods [79], [80], [81], [82] were proposed for input space UDA.…”

“…The RCCR approach [75] employs ClassMix [128] and CutMix [129] as proposed by DACS [71]. Also BAPA-Net [74] solely employs CutMiX [129].…”

“…Unlike the implicit alignment of the feature distributions, self-supervised approaches are also used for direct alignment. DACL [161], SPCL [160], UCDA [81], PWCL [162], VOLUME 11, 2023 RCCR [75], and CLST [159] make use of contrastive selfsupervised learning.…”

Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation

“…These meth- ods aim to minimize a series of adversarial losses to learn invariant representations across domains, thereby aligning source and target feature distributions. More recently, an alternative research line to reduce domain shifts focuses on building schemes based on the self-training (ST) framework [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32]. These works iteratively train the model by using both the labeled source domain data and generated pseudo-labels of unlabeled target domain data, thus achieving alignment between source and target domains.…”