Utterance Generation With Variational Auto-Encoder for Slot Filling in Spoken Language Understanding

Shin, Youhyun; Yoo, Kyung Don; Lee, Sang-goo

doi:10.1109/lsp.2019.2895284

Cited by 11 publications

(13 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparably, Variational AutoEncoder (VAE) can generate more various utterances by adding randomness to decoding conditions in both the train phase and the test phase. Thus, it is used in some data augmentation methods [12,13]. It is worth to note that all of the above methods add other parameters in the decoder to predict slot labels for generated utterances.…”

Section: Related Workmentioning

confidence: 99%

“…Most of the previous work about SLU data augmentation [8,9,10,11] focus on context augmentation and ignore the importance of different slot values. Although some methods [7,12,13] try to augment slot values and contexts simultaneously in a generative method, their augmented data do not contain many new slot values in actual, since they only use the knowledge from few training data. To aim at augmenting new slot values, we try to use pretrained language models by exploiting the latent language knowledge in these models.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models

Lin,

Xiang,

Zhou

et al. 2021

Preprint

View full text Add to dashboard Cite

Spoken Language Understanding (SLU) is one essential step in building a dialogue system. Due to the expensive cost of obtaining the labeled data, SLU suffers from the data scarcity problem. Therefore, in this paper, we focus on data augmentation for slot filling task in SLU. To achieve that, we aim at generating more diverse data based on existing data. Specifically, we try to exploit the latent language knowledge from pretrained language models by finetuning them. We propose two strategies for finetuning process: value-based and context-based augmentation. Experimental results on two public SLU datasets have shown that compared with existing data augmentation methods, our proposed method can generate more diverse sentences and significantly improve the performance on SLU.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models

Lin,

Xiang,

Zhou

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Evaluation Following previous works (Shin, Yoo, and Lee 2019;Hou et al 2018), we compute F1-score as evaluation metric with the conlleval script. 4 Implementation We built our Cluster2Cluster model with the transformer implemented by Wolf et al (2019).…”

Section: Cross Expansionmentioning

confidence: 99%

“…Following previous works (Shin, Yoo, and Lee 2019;Hou et al 2018), we conduct experiments with Bi-LSTM as slotfilling model and train it with both original training data and data augmented by different data augmentation methods. We use the same Bi-LSTM implements as previous work.…”

Section: Cross Expansionmentioning

confidence: 99%

See 1 more Smart Citation

C2C-GenDA: Cluster-to-Cluster Generation for Data Augmentation of Slot Filling

Hou¹,

Chen²,

Che³

et al. 2020

Preprint

View full text Add to dashboard Cite

Slot filling, a fundamental module of spoken language understanding, often suffers from insufficient quantity and diversity of training data. To remedy this, we propose a novel Cluster-to-Cluster generation framework for Data Augmentation (DA), named C2C-GenDA. It enlarges the training set by reconstructing existing utterances into alternative expressions while keeping semantic. Different from previous DA works that reconstruct utterances one by one independently, C2C-GenDA jointly encodes multiple existing utterances of the same semantics and simultaneously decodes multiple unseen expressions. Jointly generating multiple new utterances allows to consider the relations between generated instances and encourages diversity. Besides, encoding multiple existing utterances endows C2C with a wider view of existing expressions, helping to reduce generation that duplicates existing data. Experiments on ATIS and Snips datasets show that instances augmented by C2C-GenDA improve slot filling by 7.99 (11.9%↑) and 5.76 (13.6%↑) F-scores respectively, when there are only hundreds of training utterances. 1 * Equal contributions.

show abstract