Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation

Lee, Seungmin; Kim, Dong-Wan; Kim, Namil; Jeong, Shinkyu

doi:10.1109/iccv.2019.00018

Cited by 174 publications

(104 citation statements)

References 31 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…The CDAN [31] conditioned the adversarial training on discriminative information conveyed in the classifier predictions. By enforcing the cluster assumption, the DTA [32] utilized adversarial dropout to learn discriminative features. The GSDA [33] took the class-wise alignment, group-wise alignment and global alignment into consideration during the feature learning.…”

Section: Related Workmentioning

confidence: 99%

“…We compare our method with class-agnostic discrepancy minimization methods: RevGrad [18], [46], DAN [13], and JAN [14]. Moreover, we compare our method with the ones which explicitly or implicitly take the class information or decision boundary into consideration to learn more discriminative features: MADA [28], MCD [27], ADR [26], CDAN [31], DTA [32] and GSDA [33]. The descriptions of these methods can be found in Section 2.…”

Section: Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Contrastive Adaptation Network for Single- and Multi-Source Domain Adaptation

Kang

Jiang

Wei

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Unsupervised Domain Adaptation (UDA) makes predictions for the target domain data while manual annotations are only available in the source domain. Previous methods minimize the domain discrepancy neglecting the class information, which may lead to misalignment and poor generalization performance. To tackle this issue, this paper proposes Contrastive Adaptation Network (CAN) that optimizes a new metric named Contrastive Domain Discrepancy explicitly modeling the intra-class domain discrepancy and the inter-class domain discrepancy. To optimize CAN, two technical issues need to be addressed: 1) the target labels are not available and 2) the conventional mini-batch sampling is imbalanced. Thus we design an alternating update strategy to optimize both the target label estimations and the feature representations. Moreover, we develop class-aware sampling to enable more efficient and effective training. Our framework can be generally applied to the single-source and multi-source domain adaptation scenarios. In particular, to deal with multiple source domain data, we propose 1) multi-source clustering ensemble which exploits the complementary knowledge of distinct source domains to make more accurate and robust target label estimations, and 2) boundary-sensitive alignment to make the decision boundary better fitted to the target. Experiments are conducted on three real-world benchmarks (i.e., Office-31 and VisDA-2017 for the single-source scenario, DomainNet for the multi-source scenario). All the results demonstrate that our CAN performs favorably against the state-of-the-art methods. Ablation studies also verify the effectiveness of each key component of our proposed system.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Contrastive Adaptation Network for Single- and Multi-Source Domain Adaptation

Kang

Jiang

Wei

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…The Adversarial Discriminative Domain Adaptation (ADDA) strategy [25] follows the idea of Generative Adversarial Networks, along with discriminative modeling and untied weight sharing to learn domain-invariant features, while keeping a useful representation for the discriminative task. Drop to Adapt (DTA) [26] makes use of adversarial dropout to enforce discriminative domain-invariant features. Damodaran et al [27] proposed the Deep Joint Distribution Optimal Transport (DeepJDOT) approach, which learns both the classifier and aligned data representations between the source and target domain following a single neural framework with a loss functions based on the Optimal Transport theory [28].…”

Section: Introductionmentioning

confidence: 99%

Incremental Unsupervised Domain-Adversarial Training of Neural Networks

Gallego

Calvo-Zaragoza

Fisher

2021

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

In the context of supervised statistical learning, it is typically assumed that the training set comes from the same distribution that draws the test samples. When this is not the case, the behavior of the learned model is unpredictable and becomes dependent upon the degree of similarity between the distribution of the training set and the distribution of the test set. One of the research topics that investigates this scenario is referred to as Domain Adaptation (DA). Deep neural networks brought dramatic advances in pattern recognition and that is why there have been many attempts to provide good domain adaptation algorithms for these models. Here we take a different avenue and approach the problem from an incremental point of view, where the model is adapted to the new domain iteratively. We make use of an existing unsupervised domain-adaptation algorithm to identify the target samples on which there is greater confidence about their true label. The output of the model is analyzed in different ways to determine the candidate samples. The selected samples are then added to the source training set by self-labeling, and the process is repeated until all target samples are labeled. This approach implements a form of adversarial training in which, by moving the self-labeled samples from the target to the source set, the DA algorithm is forced to look for new features after each iteration. Our results report a clear improvement with respect to the nonincremental case in several datasets, also outperforming other state-of-the-art domain adaptation algorithms.

show abstract

“…Another successful DA approach, unsupervised domain adversarial training (Ganin et al, 2016; Tzeng et al, 2015; Lee et al, 2019; Fernando et al, 2013; Kouw and Loog, 2019; Wang and Deng, 2018) relies on domain invariant features to achieve good domain adaptation. Several adversarial training methods have been proposed, including the recent ones based on discriminator framework (Tzeng et al, 2017), partial transfer learning (Cao et al, 2018) (assuming that the target domain dataset is a subset of the source domain) and using associations between the source and target domains (e.g.…”

Section: Introductionmentioning

confidence: 99%

Comparison of domain adaptation techniques for white matter hyperintensity segmentation in brain MR images

Sundaresan

Zamboni

Dinsdale

et al. 2021

Preprint

View full text Add to dashboard Cite

Robust automated segmentation of white matter hyperintensities (WMHs) in different datasets (domains) is highly challenging due to differences in acquisition (scanner, sequence), population (WMH amount and location) and limited availability of manual segmentations to train supervised algorithms. In this work we explore various domain adaptation techniques such as transfer learning and domain adversarial learning methods, including domain adversarial neural networks and domain unlearning, to improve the generalisability of our recently proposed triplanar ensemble network, which is our baseline model. We evaluated the domain adaptation techniques on source and target domains consisting of 5 different datasets with variations in intensity profile, lesion characteristics and acquired using different scanners. For transfer learning, we also studied various training options such as minimal number of unfrozen layers and subjects required for fine-tuning in the target domain. On comparing the performance of different techniques on the target dataset, unsupervised domain adversarial training of neural network gave the best performance, making the technique promising for robust WMH segmentation.

show abstract

Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation

Cited by 174 publications

References 31 publications

Contrastive Adaptation Network for Single- and Multi-Source Domain Adaptation

Contrastive Adaptation Network for Single- and Multi-Source Domain Adaptation

Incremental Unsupervised Domain-Adversarial Training of Neural Networks

Comparison of domain adaptation techniques for white matter hyperintensity segmentation in brain MR images

Contact Info

Product

Resources

About