Unsupervised domain adaptation via coarse-to-fine feature alignment method using contrastive learning

Abstract

Previous feature alignment methods in Unsupervised domain adaptation(UDA) mostly only align global features without considering the mismatch between class-wise features. In this work, we propose a new coarse-tofine feature alignment method using contrastive learning called CFContra. It draws class-wise features closer than coarse feature alignment or class-wise feature alignment only, therefore improves the model's performance to a great extent. We build it upon one of the most effective methods of UDA called entropy minimization [47] to further improve performance. In particular, to prevent excessive memory occupation when applying contrastive loss in semantic segmentation, we devise a new way to build and update the memory bank. In this way, we make the algorithm more efficient and viable with limited memory. Extensive experiments show the effectiveness of our method and model trained on the GTA5 [39] to Cityscapes dataset has boost mIOU by 3.5 compared to the MinEnt algorithm [47]. Our code will be publicly available.

Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving

“…The style transfer is performed using the standard AdaIN method. Also, the CFContra method by Tang et al [87] employs an encoder-decoder network with standard AdaIN layers for style transfer.…”

“…Apart from the implicit adaptation through self-supervised learning and the construction of semantic pairs in the source and target domain, one can identify a third class of self-supervised domain adaptation approaches. Semantic self-supervised approaches as presented in DANCE [178] CAM [166], CFContra [87], SCDA [167], BAPA-Net [74], SWLS [53], and SSS+ST [165] which all aim to cluster the pre-logit feature space towards so-called class prototypes directly. These class prototypes are vectors that represent the pre-logit feature representations of their respective class.…”

Domain Adaptive Semantic Segmentation via Regional Contrastive Consistency Regularization

“…Great progress in contrastive learning [1,8,31,40,51,59,68,68,[71][72][73] has been achieved by encouraging the positive pairs to get closer and pulling the negative pairs apart. For semantic segmentation tasks, [1,68,73] are proposed to fit the dense pixel prediction requirements.…”

“…For semantic segmentation tasks, [1,68,73] are proposed to fit the dense pixel prediction requirements. The definition of positive pairs and negative pairs can be various, and [59,68] treated the same category samples as the positive pairs and others as the negative pairs. [40] divided the positive pairs and negative pairs according to the label distribution similarity between different patches.…”

“…The intuition of this sampling strategy is similar to the projector head, aiming to prevent from overlooking too much regional consistency for cross-domain segmentation. Secondly, we filter the negative embeddings that have the same label or pseudo label with E cut(i,j) due to the fact that some works [59,68] have proved that gathering the embeddings of the same category can be beneficial to semantic segmentation. Note that we do not enlarge the embedding distance for the same category.…”