Data Augmentation for Spoken Language Understanding via Joint Variational Generation

Yoo, Kyung Don; Shin, Youhyun; Lee, Sang-goo

doi:10.1609/aaai.v33i01.33017402

Cited by 71 publications

(24 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Therefore, the reliability of NLU becomes a precursor for the success of dialog systems. Recently, various deep neural network based NLU models are proposed and some of these models have been applied in real-world applications [1]- [3]. Most existing neural NLU modules are built by following a closed-world assumption [4], [5], i.e, the data used in the training and testing phrase are drawn from the same distribution.…”

Section: Introductionmentioning

confidence: 99%

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

Zheng

Chen

Huang

2020

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Natural Language Understanding (NLU) is a vital component of dialogue systems, and its ability to detect Out-of-Domain (OOD) inputs is critical in practical applications, since the acceptance of the OOD input that is unsupported by the current system may lead to catastrophic failure. However, most existing OOD detection methods rely heavily on manually labeled OOD samples and cannot take full advantage of unlabeled data. This limits the feasibility of these models in practical applications.In this paper, we propose a novel model to generate highquality pseudo OOD samples that are akin to IN-Domain (IND) input utterances, and thereby improves the performance of OOD detection. To this end, an autoencoder is trained to map an input utterance into a latent code. and the codes of IND and OOD samples are trained to be indistinguishable by utilizing a generative adversarial network. To provide more supervision signals, an auxiliary classifier is introduced to regularize the generated OOD samples to have indistinguishable intent labels. Experiments show that these pseudo OOD samples generated by our model can be used to effectively improve OOD detection in NLU. Besides, we also demonstrate that the effectiveness of these pseudo OOD data can be further improved by efficiently utilizing unlabeled data.

show abstract

Section: Introductionmentioning

confidence: 99%

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

Zheng

Chen

Huang

2020

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Due to the lack of high-quality SLU data, a data augmentation approach is quite essential [15]. SpecAugment [10,16] is a simple yet effective data augmentation method for E2E ASR, resulting in consistent accuracy gain.…”

Section: Data Augmentation For Slumentioning

confidence: 99%

Two-Stage Textual Knowledge Distillation for End-to-End Spoken Language Understanding

Kim

Shin

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

End-to-end approaches open a new way for more accurate and efficient spoken language understanding (SLU) systems by alleviating the drawbacks of traditional pipeline systems. Previous works exploit textual information for an SLU model via pre-training with automatic speech recognition or finetuning with knowledge distillation. To utilize textual information more effectively, this work proposes a two-stage textual knowledge distillation method that matches utterancelevel representations and predicted logits of two modalities during pre-training and fine-tuning, sequentially. We use vq-wav2vec BERT as a speech encoder because it captures general and rich features. Furthermore, we improve the performance, especially in a low-resource scenario, with data augmentation methods by randomly masking spans of discrete audio tokens and contextualized hidden representations. Consequently, we push the state-of-the-art on the Fluent Speech Commands, achieving 99.7% test accuracy in the full dataset setting and 99.5% in the 10% subset setting. Throughout the ablation studies, we empirically verify that all used methods are crucial to the final performance, providing the best practice for spoken language understanding. Code is available at https://github.com/clovaai/textual-kd-slu.

show abstract

“…The above results suggest the SGP-DST model overfits on the schema element representations and writing style in the training set, reducing its ability to generalize to new writing styles. Data augmentation is a common way to improve robustness (Hou et al 2018;Yoo, Shin, and Lee 2019). To this end, we experiment with a simple back-translation approach (Sennrich, Haddow, and Birch 2016) for augmenting the training schemas for the SGP-DST model and study its impact on model performance and schema sensitivity.…”

Section: Schema Augmentationmentioning

confidence: 99%

SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems

Lee,

Gupta,

Rastogi

et al. 2021

Preprint

View full text Add to dashboard Cite

Zero/few-shot transfer to unseen services is a critical challenge in task-oriented dialogue research. The Schema-Guided Dialogue (SGD) dataset introduced a paradigm for enabling models to support an unlimited number of services without additional data collection or re-training through the use of schemas. Schemas describe service APIs in natural language, which models consume to understand the services they need to support. However, the impact of the choice of language in these schemas on model performance remains unexplored. We address this by releasing SGD-X, a benchmark for measuring the robustness of dialogue systems to linguistic variations in schemas. SGD-X extends the SGD dataset with crowdsourced variants for every schema, where variants are semantically similar yet stylistically diverse. We evaluate two dialogue state tracking models on SGD-X and observe that neither generalizes well across schema variations, measured by joint goal accuracy and a novel metric for measuring schema sensitivity. Furthermore, we present a simple modelagnostic data augmentation method to improve schema robustness and zero-shot generalization to unseen services.

show abstract

Data Augmentation for Spoken Language Understanding via Joint Variational Generation

Cited by 71 publications

References 20 publications

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems

Two-Stage Textual Knowledge Distillation for End-to-End Spoken Language Understanding

SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems

Contact Info

Product

Resources

About