Domain-Symmetric Networks for Adversarial Domain Adaptation

Abstract

Unsupervised domain adaptation aims to learn a model of classifier for unlabeled samples on the target domain, given training data of labeled samples on the source domain. Impressive progress is made recently by learning invariant features via domain-adversarial training of deep networks. In spite of the recent progress, domain adaptation is still limited in achieving the invariance of feature distributions at a finer category level. To this end, we propose in this paper a new domain adaptation method called Domain-Symmetric Networks (SymNets). The proposed SymNet is based on a symmetric design of source and target task classifiers, based on which we also construct an additional classifier that shares with them its layer neurons. To train the SymNet, we propose a novel adversarial learning objective whose key design is based on a two-level domain confusion scheme, where the category-level confusion loss improves over the domain-level one by driving the learning of intermediate network features to be invariant at the corresponding categories of the two domains. Both domain discrimination and domain confusion are implemented based on the constructed additional classifier. Since target samples are unlabeled, we also propose a scheme of cross-domain training to help learn the target classifier. Careful ablation studies show the efficacy of our proposed method. In particular, based on commonly used base networks, our SymNets achieve the new state of the art on three benchmark domain adaptation datasets.

Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling

“…On the ImageCLEF-DA dataset (Table 3), the proposed SPL approach performs the best or the second-best in four out of six tasks and ranks the first with the average accuracy of 90.5% followed by the deep learning model SymNets (Zhang et al 2019, 89.9%) and deep feature based model MEDA (Wang et al 2018, 89.0%). On the Office-Home dataset (Table 4, again, our approach SPL outperforms all state-of-the-art models with an average accuracy of 71.0% against 70.6% by CAPLS (Wang, Bu, and Breckon 2019) and 67.6% by SymNets (Zhang et al 2019) and TADA .…”

“…Although these models can learn domain invariant features which are also discriminative for the source domain, the separability of target samples is not guaranteed since the conditional distributions are not explicitly aligned. More recently, domain-symmetric networks were proposed to promote the alignment of joint distributions of feature and category across source and target domains (Zhang et al 2019). In contrast to these approaches, pseudo-labeling target samples is another effective way to promote the alignment of conditional distributions.…”

“…ImageCLEF-DA (Caputo et al 2014) consists of four domains. We follow the existing works (Zhang et al 2019) using three of them in our experiments: Caltech-256 (C), ImageNet ILSVRC 2012 (I), and Pascal VOC 2012 (P). There are 12 classes and 50 images for each class in each domain.…”

Deep Domain Adaptation Based on Adversarial Network With Graph Regularization

“…In recent years, some scholars have carried out research on domain discriminator, for example, the discriminator can be optimized by using a gradient inversion layer in [23], Margin Disparity Discrepancy (MDD) is proposed to solve the distribution comparison with the asymmetric margin loss [24], and Batch Spectral Penalization (BSP) is proposed to boost the feature discriminability [25]. In addition, through improving single domain discriminator into multiple domain discriminators, the features of different levels [26] and the features of different classes [27][28] can be aligned more accurately, and the domain discriminator can be also designed based on two-level domain confusion scheme [29]. The method is still effective when the number of classes in the source domain is more than in the target domains [30][31], and the number of classes in the target domain is more than in the source domains [32].…”

A New Transfer Learning Fault Diagnosis Method Using TSC and JGSA Under Variable Condition

“…A new unsupervised domain adaptation approach called Collaborative and Adversarial Network (CAN) is proposed through domain-collaborative and domain-adversarial training of neural networks [31]. A new domain adaptation method called Domain-Symmetric Networks (SymNets) is proposed [32]. Transferrable Prototypical Networks (TPN) is presented for adaptation such that the prototypes for each class in the two domains are close in the embedding space and the score distributions predicted by prototypes separately on source and target data are similar [33].…”