Zero-Shot Adaptive Transfer for Conversational Language Understanding

Lee, Sung‐Jin; Jha, Rahul

doi:10.1609/aaai.v33i01.33016642

Cited by 55 publications

(59 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baselines: We compare with two strong zeroshot baselines: Zero-shot Adaptive Transfer (ZAT) (Lee and Jha, 2018) and Concept Tagger We sample positive and negative instances (Figure 3) in a ratio of 1:3. Slot values input during training and evaluation are randomly picked from values taken by the input slot in the relevant domain's training set, excluding ones that are also present in the evaluation set.…”

Section: Datasets and Experimentsmentioning

confidence: 99%

Robust Zero-Shot Cross-Domain Slot Filling with Example Values

Shah¹,

Gupta²,

Fayazi³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Task-oriented dialog systems increasingly rely on deep learning-based slot filling models, usually needing extensive labeled training data for target domains. Often, however, little to no target domain training data may be available, or the training and target domain schemas may be misaligned, as is common for web forms on similar websites. Prior zero-shot slot filling models use slot descriptions to learn concepts, but are not robust to misaligned schemas. We propose utilizing both the slot description and a small number of examples of slot values, which may be easily available, to learn semantic representations of slots which are transferable across domains and robust to misaligned schemas. Our approach outperforms state-ofthe-art models on two multi-domain datasets, especially in the low-data setting.

show abstract

Section: Datasets and Experimentsmentioning

confidence: 99%

Robust Zero-Shot Cross-Domain Slot Filling with Example Values

Shah¹,

Gupta²,

Fayazi³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Label semantics provide contextual signals that can improve model performance in multi-task and low-resource scenarios. Multiple works show that conditioning input representations on slot description embeddings improves multidomain slot labeling performance (Bapna et al, 2017;Lee and Jha, 2019). Embedding example slot values in addition to slot descriptions yields further improvements in zero-shot slot labeling (Shah et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

Augmented Natural Language for Generative Sequence Labeling

Athiwaratkun¹,

Santos²,

Krone³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We propose a generative framework for joint sequence labeling and sentence-level classification. Our model performs multiple sequence labeling tasks at once using a single, shared natural language output space. Unlike prior discriminative methods, our model naturally incorporates label semantics and shares knowledge across tasks. Our framework is general purpose, performing well on fewshot, low-resource, and high-resource tasks. We demonstrate these advantages on popular named entity recognition, slot labeling, and intent classification benchmarks. We set a new state-of-the-art for few-shot slot labeling, improving substantially upon the previous 5-shot (75.0% ! 90.9%) and 1-shot (70.4% ! 81.0%) state-of-the-art results. Furthermore, our model generates large improvements (46.27% ! 63.83%) in low-resource slot labeling over a BERT baseline by incorporating label semantics. We also maintain competitive results on high-resource tasks, performing within two points of the state-of-theart on all tasks and setting a new state-of-theart on the SNIPS dataset.

show abstract

“…• Concept Tagger (CT) [35] and Zero-shot Adaptive Transfer (ZAT) [36]: These two zero-shot methods also condition slot filling on textual slot descriptions, which have a contextual BLSTM layer and slot dependent BLSTM layer. However, they predict an IOB sequence for each slot, which may lead to overlapped segmentation and cost more time.…”

Section: B Baselinesmentioning

confidence: 99%

Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding

Zhu

Zhao

et al. 2020

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Traditional slot filling in natural language understanding (NLU) predicts a one-hot vector for each word. This form of label representation lacks semantic correlation modelling, which leads to severe data sparsity problem, especially when adapting an NLU model to a new domain. To address this issue, a novel label embedding based slot filling framework is proposed in this paper. Here, distributed label embedding is constructed for each slot using prior knowledge. Three encoding methods are investigated to incorporate different kinds of prior knowledge about slots: atomic concepts, slot descriptions, and slot exemplars. The proposed label embeddings tend to share text patterns and reuses data with different slot labels. This makes it useful for adaptive NLU with limited data. Also, since label embedding is independent of NLU model, it is compatible with almost all deep learning based slot filling models. The proposed approaches are evaluated on three datasets. Experiments on single domain and domain adaptation tasks show that label embedding achieves significant performance improvement over traditional one-hot label representation as well as advanced zero-shot approaches.

show abstract

Zero-Shot Adaptive Transfer for Conversational Language Understanding

Cited by 55 publications

References 11 publications

Robust Zero-Shot Cross-Domain Slot Filling with Example Values

Robust Zero-Shot Cross-Domain Slot Filling with Example Values

Augmented Natural Language for Generative Sequence Labeling

Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding

Contact Info

Product

Resources

About