CRUISE: Cold-Start New Skill Development via Iterative Utterance Generation

Shen, Yilin; Ray, Avik; Patel, Abhishek; Jin, Hongxia

doi:10.18653/v1/p18-4018

Cited by 8 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, most of these toolkits require both linguistic expertise and a large amount of annotated data. CRUISE (Shen et al, 2018) provide an utterance generation system to reduce the human workload of data annotation. However, CRUISE focuses on spoken language understanding.…”

Section: Related Workmentioning

confidence: 99%

EUSP: An Easy-to-Use Semantic Parsing PlatForm

Chen²,

Han

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Semantic parsing aims to map natural language utterances into structured meaning representations. We present a modular platform, EUSP (Easy-to-Use Semantic Parsing PlatForm), that facilitates developers to build semantic parser from scratch. Instead of requiring a large amount of training data or complex grammar knowledge, in our platform developers can build grammar-based semantic parser or neural-based semantic parser through configure files which specify the modules and components that compose semantic parsing system. A high quality grammar-based semantic parsing system only requires domain lexicons rather than costly training data for a semantic parser. Furthermore, we provide a browser-based method to generate the semantic parsing system to minimize the difficulty of development. Experimental results show that the neural-based semantic parser system achieves competitive performance on semantic parsing task, and grammar-based semantic parsers significantly improve the performance of a business search engine.

show abstract

Section: Related Workmentioning

confidence: 99%

EUSP: An Easy-to-Use Semantic Parsing PlatForm

Chen²,

Han

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…airline reservation), it can be both time-consuming and expensive to collect and annotate training utterances corresponding to each possible combination of slots. Secondly, for resource constrained cold-start skill developers [11], it is cheaper and easier to annotate a small number of short utterances (with just one or two slots) for training, than longer utterances with many slots which the SLU model may encounter after deployment. Building compositional SLU models which can generalize well under both these settings is vital for both scalable development, and reliability of future AI agents.…”

Section: Introductionmentioning

confidence: 99%

Compositional Generalization in Spoken Language Understanding

Ray¹,

Shen²,

Jin³

2023

Interspeech 2023

View full text Add to dashboard Cite

State-of-the-art spoken language understanding (SLU) models have shown tremendous success in benchmark SLU datasets, yet they still fail in many practical scenario due to the lack of model compositionality when trained on limited training data. In this paper, we study two types of compositionality: novel slot combination, and length generalization. We first conduct in-depth analysis, and find that state-of-the-art SLU models often learn spurious slot correlations during training, which leads to poor performance in both compositional cases. To mitigate these limitations, we create the first compositional splits of benchmark SLU datasets and we propose the first compositional SLU model, including compositional loss and paired training that tackle each compositional case respectively. On both benchmark and compositional splits in ATIS and SNIPS, we show that our compositional SLU model significantly outperforms (up to 5% F1 score) state-of-the-art BERT SLU model.

show abstract

“…To address the challenges in open-world settings, previous works adopt varied strategies. Shen et al (2018aShen et al ( , 2019c) use a cold-start algorithm to generate additional training data to cover a larger variety of utterances. This strategy relies on the software developers to pre-build all possible skills.…”

Section: Introductionmentioning

confidence: 99%

Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU

Shen

Hsu

Ray

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Intent classification is a major task in spoken language understanding (SLU). Since most models are built with pre-collected in-domain (IND) training utterances, their ability to detect unsupported out-of-domain (OOD) utterances has a critical effect in practical use. Recent works have shown that using extra data and labels can improve the OOD detection performance, yet it could be costly to collect such data. This paper proposes to train a model with only IND data while supporting both IND intent classification and OOD detection. Our method designs a novel domain-regularized module (DRM) to reduce the overconfident phenomenon of a vanilla classifier, achieving a better generalization in both cases. Besides, DRM can be used as a drop-in replacement for the last layer in any neural network-based intent classifier, providing a low-cost strategy for a significant improvement. The evaluation on four datasets shows that our method built on BERT and RoBERTa models achieves state-of-the-art performance against existing approaches and the strong baselines we created for the comparisons.

show abstract

CRUISE: Cold-Start New Skill Development via Iterative Utterance Generation

Cited by 8 publications

References 11 publications

EUSP: An Easy-to-Use Semantic Parsing PlatForm

EUSP: An Easy-to-Use Semantic Parsing PlatForm

Compositional Generalization in Spoken Language Understanding

Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU

Contact Info

Product

Resources

About