SC-UDA: Style and Content Gaps aware Unsupervised Domain Adaptation for Object Detection

Yu, F. Richard; Wang, Di; Chen, Yinpeng; Karianakis, Nikolaos; Shen, Tong; Yu, Pei; Lymberopoulos, Dimitrios; Lu, Sidi; Shi, Wei; Chen, Xiang

doi:10.1109/wacv51458.2022.00113

Cited by 26 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These strategies can be applied at different feature extraction stages of the object detection model. Data manipulation-based methods directly augment (Prakash et al, 2019;Wang et al, 2021a) or perform style transformations (Yun et al, 2021;Yu et al, 2022) on input data to narrow the distribution gap between the source and target domains. Learning strategy-based methods achieve object detection of target domain by introducing some learning strategies like self-training (Zhao et al, 2020a;Li et al, 2021) and teacher-student networks (He et al, 2022;Li et al, 2022b).…”

Section: Domain Adaptive Object Detectionmentioning

confidence: 99%

Frequency Domain Disentanglement for Arbitrary Neural Style Transfer

Luo

Wang

et al. 2023

AAAI

View full text Add to dashboard Cite

Arbitrary neural style transfer has been a popular research topic due to its rich application scenarios. Effective disentanglement of content and style is the critical factor for synthesizing an image with arbitrary style. The existing methods focus on disentangling feature representations of content and style in the spatial domain where the content and style components are innately entangled and difficult to be disentangled clearly. Therefore, these methods always suffer from low-quality results because of the sub-optimal disentanglement. To address such a challenge, this paper proposes the frequency mixer (FreMixer) module that disentangles and re-entangles the frequency spectrum of content and style components in the frequency domain. Since content and style components have different frequency-domain characteristics (frequency bands and frequency patterns), the FreMixer could well disentangle these two components. Based on the FreMixer module, we design a novel Frequency Domain Disentanglement (FDD) framework for arbitrary neural style transfer. Qualitative and quantitative experiments verify that the proposed method can render better stylized results compared to the state-of-the-art methods.

show abstract

Section: Domain Adaptive Object Detectionmentioning

confidence: 99%

Frequency Domain Disentanglement for Arbitrary Neural Style Transfer

Luo

Wang

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…In view of above problems, recent researchers have focused their efforts on Unsupervised Domain Adaption (UDA) methods [16][17][18][19][20][21][22], which leverages unsupervised transfer learning to alleviate the domain gaps. UDA methods transfer knowledge from the labelrich source domain to the target domain without tedious manual annotations.…”

Section: Introductionmentioning

confidence: 99%

“…To some extent, the quality of pseudo labels are tightly related to the detection precision. To avoid the noise of the pseudo labels, many novel self-training optimizations [13,19,20,22,[30][31][32] are proposed, including knowledge distillation strategy [19], the progressive confidence restriction [13], imbalanced mini-batch sampling strategy [20], and graph representation [22,32]. Although self-training strategy is an efficient way to boost performance, one shortcoming of these methods is that classification confidences are mostly used as the prediction box selection criteria.…”

Section: Introductionmentioning

confidence: 99%

AdvMix: Adversarial Mixing Strategy for Unsupervised Domain Adaptive Object Detection

Chen,

Lv,

Dai

et al. 2024

Electronics

View full text Add to dashboard Cite

Recent object detection networks suffer from performance degradation when training data and test data are distinct in image styles and content distributions. In this paper, we propose a domain adaptive method, Adversarial Mixing (AdvMix), where the label-rich source domain and unlabeled target domain are jointly trained by the adversarial feature alignment and a self-training strategy. To diminish the style gap, we design the Adversarial Gradient Reversal Layer (AdvGRL), containing a global-level domain discriminator to align the domain features by gradient reversal, and an adversarial weight mapping function to enhance the stability of domain-invariant features by hard example mining. To eliminate the content gap, we introduce a region mixing self-supervised training strategy where a region of the target image with the highest confidence is selected to merge with the source image, and the synthesis image is self-supervised by the consistency loss. To improve the reliability of self-training, we propose a strict confidence metric combining both object and bounding box uncertainty. Extensive experiments conducted on three benchmarks demonstrate that AdvMix achieves prominent performance in terms of detection accuracy, surpassing existing domain adaptive methods by nearly 5% mAP.

show abstract

“…Most DA detection models aim to measure the feature distribution distance of different domains and then minimize the discrepancy, therefore adversarial manner is exploited between the feature extractor and domain discriminator (Chen et al, 2018;Saito et al, 2019;Li et al, 2020;Chen et al, 2021). Adversarial feature learning aims to decrease style gaps (e.g., color, texture, illumination) between domains for improving the generalization ability, however, for the challenges of content gaps such as various locations, densities, and distributions, the adversarial feature learning may lead to feature misalignment, which decreases the discriminability of the detector (Jiang et al, 2022;Yu et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

Soft-masks guided faster region-based convolutional neural network for domain adaptation in wind turbine detection

Luo

Yuan

et al. 2023

Front. Energy Res.

View full text Add to dashboard Cite

Wind turbine generator system plays a fundamental role in electricity generation in industry 4.0, and wind turbines are usually distributed separately and in poor locations. Unmanned Aerial Vehicles (UAV) which could overcome the above challenges are deployed to collect photographs of wind turbines, could be used for predictive maintenance of wind turbines and energy management. However, identifying meaningful information from huge amounts of photographs taken by drones is a challenging task due to various scales, different viewpoints, and tedious manual annotation. Besides, deep neural networks (DNN) are dominant in object detection, and training DNN requires large numbers of accurately labeled training data, and manual data annotation is tedious, inefficient, and error-prone. Considering these issues, we generate a synthetic UAV-taken dataset of wind turbines, which provides RGB images, target bounding boxes, and precise pixel annotations as well. But directly transferring the model trained on the synthetic dataset to the real dataset may lead to poor performance due to domain shifts (or domain gaps). The predominant approaches to alleviate the domain discrepancy are adversarial feature learning strategies, which focus on feature alignment for style (e.g., color, texture, illumination, etc.) gaps without considering the content (e.g., densities, backgrounds, and layout scenes) gaps. In this study, we scrutinize the real UAV-taken imagery of wind turbines and develop a synthetic generation method trying to simulate the real ones from the aspects of style and content. Besides, we propose a novel soft-masks guided faster region-based convolutional neural network (SMG Faster R-CNN) for domain adaptation in wind turbine detection, where the soft masks help to extract highly object-related features and suppress domain-specific features. We evaluate the accuracy of SMG Faster R-CNN on the wind turbine dataset and demonstrate the effectiveness of our approach compared with some prevalent object detection models and some adversarial DA models.

show abstract

SC-UDA: Style and Content Gaps aware Unsupervised Domain Adaptation for Object Detection

Cited by 26 publications

References 21 publications

Frequency Domain Disentanglement for Arbitrary Neural Style Transfer

Frequency Domain Disentanglement for Arbitrary Neural Style Transfer

AdvMix: Adversarial Mixing Strategy for Unsupervised Domain Adaptive Object Detection

Soft-masks guided faster region-based convolutional neural network for domain adaptation in wind turbine detection

Contact Info

Product

Resources

About