Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Wang, Liwen; Li, Xuefeng; Liu, Jiachi; He, Keqing; Yan, Yuanmeng; Wang, Xu

doi:10.18653/v1/2021.emnlp-main.746

Cited by 14 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prototypical contrastive learning and label confusion (PCLC) [18]: PCLC is a pipeline approach using contrastive learning and label confusion algorithm for improving the predictions of unseen slots. PCLC achieved state-ofthe-art performance on the Snips dataset by using modified slot name descriptions.…”

Section: Baseline Methodsmentioning

confidence: 99%

“…Pipeline methods [17], [18] were proposed to learn context-based representations to improve the prediction of complete slot entities. A context-based representation captures the general characteristics of slot entities according to contextual information.…”

Section: Multi-relation-based Representationmentioning

confidence: 99%

“…For fair comparison, we used the same version of Snips as previous studies. [15]- [18]. SGD: SGD [21] is an unprecedented large-scale dataset with 20 domains including a variety of well-encountered scenario in the real-world task-oriented dialogue system applications, such as setting alarm, booking tickets or hotels, renting cars, and asking for weathers.…”

Section: Datasetsmentioning

confidence: 99%

“…Most previous studies achieved zero-shot slot filling by predicting both slot entities and slot types and then combining the predictions as outputs. Those methods can be divided into two types according to the prediction process: end-to-end machine reading comprehension (MRC) methods [14]- [16] and two-step pipeline methods [17], [18]. The MRC methods predict slot entities for each slot type in order to handle unseen slots with fixed model parameters, whereas the pipeline methods predict slot entities and slot types separately in order to focus on improving the slot entity predictions.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Intrinsic Representation Mining for Zero-Shot Slot Filling

Okada

Dang

2022

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Zero-shot slot filling is a domain adaptation approach to handle unseen slots in new domains without training instances. Previous studies implemented zero-shot slot filling by predicting both slot entities and slot types. Because of the lack of knowledge about new domains, the existing methods often fail to predict slot entities for new domains as well as cannot effectively predict unseen slot types even when slot entities are correctly identified. Moreover, for some seen slot types, those methods may suffer from the domain shift problem, because the unseen context in new domains may change the explanations of the slots. In this study, we propose intrinsic representations to alleviate the domain shift problems above. Specifically, we propose a multi-relation-based representation to capture both the general and specific characteristics of slot entities, and an ontology-based representation to provide complementary knowledge on the relationships between slots and values across domains, for handling both unseen slot types and unseen contexts. We constructed a two-step pipeline model using the proposed representations to solve the domain shift problem. Experimental results in terms of the F1 score on three large datasets-Snips, SGD, and MultiWOZ 2.3-showed that our model outperformed state-of-the-art baselines by 29.62, 10.38, and 3.89, respectively. The detailed analysis with the average slot F1 score showed that our model improved the prediction by 25.82 for unseen slot types and by 10.51 for seen slot types. The results demonstrated that the proposed intrinsic representations can effectively alleviate the domain shift problem for both unseen slot types and seen slot types with unseen contexts.

show abstract

Section: Baseline Methodsmentioning

confidence: 99%

Section: Multi-relation-based Representationmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Intrinsic Representation Mining for Zero-Shot Slot Filling

Okada

Dang

2022

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…To separate the text features in different domains, we apply the supervised contrastive learning [11] [12] to train the domain feature extractor. We consider the samples from the same domain as positive samples and samples from other domains as the negative samples to keep the text feature of the same domain together and away from other features.…”

Section: Domain Supervised Contrastive Learningmentioning

confidence: 99%

A Robust Contrastive Alignment Method For Multi-Domain Text Classification

Li¹,

Liang²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Multi-domain text classification can automatically classify texts in various scenarios. Due to the diversity of human languages, texts with the same label in different domains may differ greatly, which brings challenges to the multidomain text classification. Current advanced methods use the private-shared paradigm, capturing domain-shared features by a shared encoder, and training a private encoder for each domain to extract domain-specific features. However, in realistic scenarios, these methods suffer from inefficiency as new domains are constantly emerging. In this paper, we propose a robust contrastive alignment method to align text classification features of various domains in the same feature space by supervised contrastive learning. By this means, we only need two universal feature extractors to achieve multidomain text classification. Extensive experimental results show that our method performs on par with or sometimes better than the state-of-the-art method, which uses the complex multi-classifier in a private-shared framework.

show abstract

Improving Cross-Domain Named Entity Recognition from the Perspective of Representation

Cai

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Cited by 14 publications

References 22 publications

Intrinsic Representation Mining for Zero-Shot Slot Filling

Intrinsic Representation Mining for Zero-Shot Slot Filling

A Robust Contrastive Alignment Method For Multi-Domain Text Classification

Improving Cross-Domain Named Entity Recognition from the Perspective of Representation

Contact Info

Product

Resources

About