WIND: Weighting Instances Differentially for Model-Agnostic Domain Adaptation

Chen, Xiang; Cao, Yue; Wan, Xiaojun

doi:10.18653/v1/2021.findings-acl.209

Cited by 4 publications

(8 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To reduce the computation cost, we use the approximation technique in (Chen et al, 2021) to compute the training guidance (i.e., ∂L M ( θ(w))…”

Section: Meta-learning Modulementioning

confidence: 99%

“…Comparing Methods Since the DaMSTF can be customized to both semi-supervised and unsupervised domain adaptation scenarios, the baselines contain both unsupervised and semisupervised domain adaptation approaches. For the unsupervised domain adaptation, Out (Chen et al, 2021), DANN (Ganin et al, 2016) and CRST (Zou et al, 2019) are selected as the baselines, while In+Out (Chen et al, 2021), MME (Saito et al, 2019), BiAT (Jiang et al, 2020), and Wind (Chen et al, 2021) are selected as the baselines for the semi-supervised domain adaptation. Out and In+Out are two straightforward ways for realizing unsupervised and semi-supervised domain adaptation, where Out means the base model is trained on the out-of-domain data (i.e., labeled source domain data) and In+Out means the base model is trained on both the in-domain and the out-of-domain data.…”

Section: Experiments Settingsmentioning

confidence: 99%

“…Domain adaptation, which aims to adapt the model trained on the source domain to the target domain, † contributed equally to this work * corresponding author attracts much attention in Natural Language Processing (NLP) applications (Du et al, 2020;Chen et al, 2021;Lu et al, 2022). Since domain adaptation involves labeled data from the source domain and unlabeled data from the target domain, it can be regarded as a semi-supervised learning problem.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DaMSTF: Domain Adversarial Learning Enhanced Meta Self-Training for Domain Adaptation

Lu¹,

Huang²,

Zhao³

et al. 2023

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Self-training emerges as an important research line on domain adaptation. By taking the model's prediction as the pseudo labels of the unlabeled data, self-training bootstraps the model with pseudo instances in the target domain. However, the prediction errors of pseudo labels (label noise) challenge the performance of self-training. To address this problem, previous approaches only use reliable pseudo instances, i.e., pseudo instances with high prediction confidence, to retrain the model. Although these strategies effectively reduce the label noise, they are prone to miss the hard examples. In this paper, we propose a new self-training framework for domain adaptation, namely Domain adversarial learning enhanced Self-Training Framework (DaMSTF). Firstly, DaMSTF involves meta-learning to estimate the importance of each pseudo instance, so as to simultaneously reduce the label noise and preserve hard examples. Secondly, we design a meta constructor for constructing the meta validation set, which guarantees the effectiveness of the meta-learning module by improving the quality of the meta validation set. Thirdly, we find that the meta-learning module suffers from the training guidance vanishment and tends to converge to an inferior optimal. To this end, we employ domain adversarial learning as a heuristic neural network initialization method, which can help the meta-learning module converge to a better optimal. Theoretically and experimentally, we demonstrate the effectiveness of the proposed DaMSTF. On the cross-domain sentiment classification task, DaMSTF improves the performance of BERT with an average of nearly 4%.

show abstract

“…To reduce the computation cost, we use the approximation technique in (Chen et al, 2021) to compute the training guidance (i.e., ∂L M ( θ(w))…”

Section: Meta-learning Modulementioning

confidence: 99%

Section: Experiments Settingsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DaMSTF: Domain Adversarial Learning Enhanced Meta Self-Training for Domain Adaptation

Lu¹,

Huang²,

Zhao³

et al. 2023

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

show abstract

“…Methods for cross-domain text classification can be roughly categorized into two classes: taskagnostic methods, and pivot-based methods. The former includes divergence minimization [16,18,10], stacked denoising auto-encoders [9], discriminative adversarial training [7,20], instance reweighting [3] and so forth. There are also some works that combine task-agnostic methods with NLP-specific approaches or models [8,6].…”

Section: Cross-domain Text Classificationmentioning

confidence: 99%

“…The most prominent pivot-based methods are Structure Correspondence Learning (SCL) and its variants [2,25,27]. In SCL, the pivots are defined as the words that occur frequently on both source and target domains and behave in similar ways that are discriminable for the classification task 3 . The model can effectively learn domain-invariant features for pivots, but it is more challenging for the non-pivots as they have domain-specific meanings.…”

Section: Introductionmentioning

confidence: 99%

Domain-Adaptive Text Classification with Structured Knowledge from Unlabeled Data

Li¹,

Chen²,

Dong³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Domain adaptive text classification is a challenging problem for the large-scale pretrained language models because they often require expensive additional labeled data to adapt to new domains. Existing works usually fails to leverage the implicit relationships among words across domains. In this paper, we propose a novel method, called Domain Adaptation with Structured Knowledge (DASK), to enhance domain adaptation by exploiting word-level semantic relationships. DASK first builds a knowledge graph to capture the relationship between pivot terms (domain-independent words) and non-pivot terms in the target domain. Then during training, DASK injects pivot-related knowledge graph information into source domain texts. For the downstream task, these knowledge-injected texts are fed into a BERT variant capable of processing knowledge-injected textual data. Thanks to the knowledge injection, our model learns domain-invariant features for non-pivots according to their relationships with pivots. DASK ensures the pivots to have domain-invariant behaviors by dynamically inferring via the polarity scores of candidate pivots during training with pseudo-labels. We validate DASK on a wide range of cross-domain sentiment classification tasks and observe up to 2.9% absolute performance improvement over baselines for 20 different domain pairs. Code will be made available at https://github.com/hikaru-nara/DASK.

show abstract