Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization

Wang, Shujun; Yu, Lequan; Li, Caizi; Fu, Chi‐Wing; Heng, Pheng‐Ann

doi:10.1007/978-3-030-58545-7_10

Cited by 136 publications

(84 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of ERM [49], GroupDRO [44], Mixup [58], MLDG [22], DANN [12], C-DANN [32], CORAL [47], MMD [24], IRM [3], ARM [60], MTL [5], SagNet [37], and RSC [17] are cited from [14] except for Office-31, on which we conduct experiments and report the results reproduced by us. For PACS, we also compare our method with EisNet [57], Entropy Regularization (ER) [62], DMG [7], DSON [46], MASF [9] and MetaReg [4] which also use ResNet-50 as the backbone, the results of these methods are directly cited from original papers. It is worth noting that from the results reported in previous DG research over the years, one can easily find that even a marginal improvement (e.g., 1%) on average accuracy is challenging in the DG community.…”

Section: Comparison With Other Methodsmentioning

confidence: 99%

Learning Transferrable and Interpretable Representations for Domain Generalization

Lü

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Conventional machine learning models are often vulnerable to samples with different distributions from the ones of training samples, which is known as domain shift. Domain Generalization (DG) challenges this issue by training a model based on multiple source domains and generalizing it to arbitrary unseen target domains. In spite of remarkable results made in DG, a majority of existing works lack a deep understanding of the feature representations learned in DG models, resulting in limited generalization ability when facing domains out-of-distribution. In this paper, we aim to learn a domain transformation space via a domain transformer network (DTN) which explicitly mines the relationship among multiple domains and constructs transferrable feature representations for down-stream tasks by interpreting each feature as a semantically weighted combination of multiple domain-specific features. Our DTN is encouraged to meta-learn the properties and characteristics of domains during the training process based on multiple seen domains, making transformed feature representations more semantical, thus generalizing better to unseen domains. Once the model is constructed, the feature representations of unseen target domains can also be inferred adaptively by selectively combining the feature representations from the diverse set of seen domains. We conduct extensive experiments on five DG benchmarks and the results strongly demonstrate the effectiveness of our approach. CCS CONCEPTS• Computing methodologies → Computer vision; Learning paradigms; Image representations.

show abstract

Section: Comparison With Other Methodsmentioning

confidence: 99%

Learning Transferrable and Interpretable Representations for Domain Generalization

Lü

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…DomainNet includes 345 object categories from six domains (i.e., Sketch, Real, Quickdraw, Painting, Infograph, and Clipart). Following previous works [12,8], we selected one domain as an unseen domain and used the remaining domains for training. To introduce the notion of an open set, we split all classes into three subsets: C k , C su , and As for baselines (i.e., L DG only), we used three DG methods: Deep All, JiGen [11], and MMLD [8].…”

Section: Methodsmentioning

confidence: 99%

“…In the open-set setting proposed by Panareda Busto and Gall [5], examples of unknown classes are given in both source and target domains. Since then, Saito et al [6] and You et al [7] In many DG approaches, the feature distributions among source domains are aligned by adversarial training [8], metalearning [9,10], or self-supervised learning [11,12]. In this study, we introduce OSDG, which is the combination of DG and open-set recognition in the vein of Panareda Busto and Gall [5], and design the novel loss for OSDG.…”

Section: Related Workmentioning

confidence: 99%

Open-Set Domain Generalization VIA Metric Learning

Katsumata

Kishida

Amma

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

In this study, we address open-set domain generalization, which aims to reject unknown class samples while classifying known class samples in unseen domains. Conventional domain generalization has the problem of unknown class samples being classified as known classes because domain generalization methods align feature distributions without distinction between known and unknown classes. To tackle this problem, we propose a decoupling loss that diffuses the feature representations of unknown samples. The loss allows us to construct a feature space that can better distinguish unknown samples. We demonstrate the effectiveness of decoupling loss using open-set domain generalization benchmarks.

show abstract

“…Domain augmentation is another popular approach that aims to generate out-of-distribution samples, e.g., via domain-adversarial generation [1,35] or gradient-based adversarial attacks [34,42]. Recently, some regularization methods have also been proposed to address domain generalization via shapebiased learning [27] or self-supervise methods [4,43]. Nevertheless, these methods cannot guarantee that features from unseen target domains share the same representation space learned by domain-invariant learning or regularization methods, since the learned model might be overfitting in the source domains due to the absence of target domain.…”

Section: Related Workmentioning

confidence: 99%

“…However, since collecting data from arbitrary target domains is expensive and impractical, domain generalization (DG) is proposed as a more challenging and practical problem setting, which aims to use multiple source domains to train a model that can generalize well to arbitrary unseen target domains. This setting has aroused wide attention from researchers and many works have been proposed to address DG problem, mainly via domain-invariant learning [16,29], meta-learning [2,8], domain augmentation [34,42], shape-biased learning [27,43], etc. However, these methods unavoidably face the problem of overfitting in the source domains because the target domain is inaccessible in the training procedure, i.e., the existing methods are inclined to focus on these limited source domains, which will lead to unsatisfactory generalizing ability in unseen target domain.…”

Section: Introductionmentioning

confidence: 99%

Domain Generalization via Progressive Layer-wise and Channel-wise Dropout

Guo¹,

Qi²,

Shi³

et al. 2021

Preprint

View full text Add to dashboard Cite

By training a model on multiple observed source domains, domain generalization aims to generalize well to arbitrary unseen target domains without further training. Existing works mainly focus on learning domain-invariant features to improve the generalization ability. However, since target domain is not available during training, previous methods inevitably suffer from overfitting in source domains. To tackle this issue, we develop an effective dropoutbased framework to enlarge the region of the model's attention, which can effectively mitigate the overfitting problem. Particularly, different from the typical dropout scheme, which normally conducts the dropout on the fixed layer, first, we randomly select one layer, and then we randomly select its channels to conduct dropout. Besides, we leverage the progressive scheme to add the ratio of the dropout during training, which can gradually boost the difficulty of training model to enhance the robustness of the model. Moreover, to further alleviate the impact of the overfitting issue, we leverage the augmentation schemes on image-level and featurelevel to yield a strong baseline model. We conduct extensive experiments on multiple benchmark datasets, which show our method can outperform the state-of-the-art methods.

show abstract

Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization

Cited by 136 publications

References 38 publications

Learning Transferrable and Interpretable Representations for Domain Generalization

Learning Transferrable and Interpretable Representations for Domain Generalization

Open-Set Domain Generalization VIA Metric Learning

Domain Generalization via Progressive Layer-wise and Channel-wise Dropout

Contact Info

Product

Resources

About