Domain Confused Contrastive Learning for Unsupervised Domain Adaptation

Long, Quanyu; Luo, Tianze; Wang, Wenya; Pan, Sinno Jialin

doi:10.18653/v1/2022.naacl-main.217

Cited by 11 publications

(10 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data We use the Amazon Reviews dataset (Ni et al, 2019), a dataset that facilitates research in tasks like sentiment analysis (Zhang et al, 2020), aspect-based sentiment analysis, and recommendation systems different product categories that serve as domains, which makes it a natural testbed for many multidomain studies. A noteworthy example of a research field that heavily relies on this dataset is domain adaptation (Blitzer et al, 2007;Ziser and Reichart, 2018;Du et al, 2020;Lekhtman et al, 2021;Long et al, 2022), which is the task of learning robust models across different domains, closely related to our research. 2 We sort the domains by their review counts and pick the top five, which results in: Books, Clothing Shoes and Jewelry, Electronics, Home and Kitchen, and Movies and TV domains.…”

Section: Methodsmentioning

confidence: 76%

Understanding Domain Learning in Language Models Through Subpopulation Analysis

Zhang¹,

Ziser²,

Cohen³

2022

Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

View full text Add to dashboard Cite

We investigate how different domains are encoded in modern neural network architectures. We analyze the relationship between natural language domains, model size, and the amount of training data used. The primary analysis tool we develop is based on subpopulation analysis with Singular Vector Canonical Correlation Analysis (SVCCA), which we apply to Transformer-based language models (LMs). We compare the latent representations of such a language model at its different layers from a pair of models: a model trained on multiple domains (an experimental model) and a model trained on a single domain (a control model). Through our method, we find that increasing the model capacity impacts how domain information is stored in upper and lower layers differently. In addition, we show that larger experimental models simultaneously embed domainspecific information as if they were conjoined control models. These findings are confirmed qualitatively, demonstrating the validity of our method.

show abstract

Section: Methodsmentioning

confidence: 76%

Understanding Domain Learning in Language Models Through Subpopulation Analysis

Zhang¹,

Ziser²,

Cohen³

2022

Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

View full text Add to dashboard Cite

show abstract

“…For instance, in computer vision, it can be used for tasks such as image recognition [21], object detection [22], and image segmentation [23]. In natural language processing, it can be used for tasks such as text classification [24], machine translation [25], and language modeling [26].…”

Section: Unsupervised Transfer Learningmentioning

confidence: 99%

Innovative integration of multi-scale residual networks and MK-MMD for enhanced feature representation in fault diagnosis

Li,

Yuan,

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

This article proposes a method based on multi-scale expansion of residual neural networks to address challenges in the operation of rotating components, such as bearings and gears, under complex conditions where they are often affected by environmental noise. This interference leads to weaker fault characteristics, making feature selection difficult and increasing the presence of extraneous information features. To tackle this issue, the proposed method first employs a multi-scale feature residual neural network to extract features from vibration signals of rotating machinery. The method decomposes the signal into multiple sub-signals of different scales, extracting local features at each scale. It then uses residual connections to combine these local features to obtain a global feature representation. Furthermore, the article introduces a construction of the maximum mean discrepancy and minimization of entropy boundaries to adapt to the differences between two domains. The method utilizes multiple kernel functions to calculate distances between data at different scales and combines these distances to obtain a comprehensive measure. By employing the maximum mean discrepancy and minimization of entropy boundary approach, the method can more accurately determine whether signals at different scales belong to the same category, thereby improving diagnostic accuracy and robustness. Experimental results demonstrate the effectiveness of the proposed method in unsupervised cross-domain fault diagnosis tasks. Future work will focus on further optimizing the architecture of residual neural networks, enhancing feature extraction capabilities, and exploring advanced data augmentation methods to further improve the model's generalization performance.

show abstract

“…• Model adaptation for recommendation: Given large pretrained models, it is often necessary to adapt the models to a recommendation task with domain-specific data. We will review the common paradigms for model adaptation, including representationbased transfer, fine-tuning, adapter tuning [26], prompt tuning [13], and retrieval-augmented adaptation [33].…”

Section: Multimodal Pretraining For Recommendationmentioning

confidence: 99%

Multimodal Pretraining and Generation for Recommendation: A Tutorial

Zhu,

Zhou,

et al. 2024

Companion Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

Personalized recommendation stands as a ubiquitous channel for users to explore information or items aligned with their interests. Nevertheless, prevailing recommendation models predominantly rely on unique IDs and categorical features for user-item matching. While this ID-centric approach has witnessed considerable success, it falls short in comprehensively grasping the essence of raw item contents across diverse modalities, such as text, image, audio, and video. This underutilization of multimodal data poses a limitation to recommender systems, particularly in the realm of multimedia services like news, music, and short-video platforms. The recent surge in pretraining and generation techniques presents both opportunities and challenges in the development of multimodal recommender systems. This tutorial seeks to provide a thorough exploration of the latest advancements and future trajectories in multimodal pretraining and generation techniques within the realm of recommender systems. The tutorial comprises three parts: multimodal pretraining, multimodal generation, and industrial applications and open challenges in the field of recommendation. Our target audience encompasses scholars, practitioners, and other parties interested in this domain. By providing a succinct overview of the field, we aspire to facilitate a swift understanding of multimodal recommendation and foster meaningful discussions on the future development of this evolving landscape.

show abstract

Domain Confused Contrastive Learning for Unsupervised Domain Adaptation

Cited by 11 publications

References 28 publications

Understanding Domain Learning in Language Models Through Subpopulation Analysis

Understanding Domain Learning in Language Models Through Subpopulation Analysis

Innovative integration of multi-scale residual networks and MK-MMD for enhanced feature representation in fault diagnosis

Multimodal Pretraining and Generation for Recommendation: A Tutorial

Contact Info

Product

Resources

About