Insufficient Data Can Also Rock! Learning to Converse Using Smaller Data with Augmentation

Abstract

Recent successes of open-domain dialogue generation mainly rely on the advances of deep neural networks. The effectiveness of deep neural network models depends on the amount of training data. As it is laboursome and expensive to acquire a huge amount of data in most scenarios, how to effectively utilize existing data is the crux of this issue. In this paper, we use data augmentation techniques to improve the performance of neural dialogue models on the condition of insufficient data. Specifically, we propose a novel generative model to augment existing data, where the conditional variational autoencoder (CVAE) is employed as the generator to output more training data with diversified expressions. To improve the correlation of each augmented training pair, we design a discriminator with adversarial training to supervise the augmentation process. Moreover, we thoroughly investigate various data augmentation schemes for neural dialogue system with generative models, both GAN and CVAE. Experimental results on two open corpora, Weibo and Twitter, demonstrate the superiority of our proposed data augmentation model.

Controllable Paraphrase Generation with a Syntactic Exemplar

“…This task can benefit natural language interfaces to information systems by suggesting alternative invocation phrases for particular types of queries (Kumar et al, 2017). It can also bear on dialogue systems that seek to generate utterances that fit particular functional categories (Ke et al, 2018;Li et al, 2019).…”

Data Manipulation: Towards Effective Instance Learning for Neural Dialogue Generation via Learning to Augment and Reweight

“…The transformer model for dialogue generation is configured with 512 hidden size, 8 attention heads and 6 blocks in both the encoder and decoder. The hyper-parameters in the baseline models are set following the original papers (Li et al, 2019;Shang et al, 2018;Csáky et al, 2019).…”

“…Hou et al (2018) utilize a sequence-to-sequence model to produce diverse utterances for language understanding. Li et al (2019); Niu and Bansal (2019) propose to generate sentences for dialogue augmentation. Compared with previous augmentation approaches for dialogue generation, augmented sentences in our framework are selectively generated using the pretrained models and the augmentation process is additionally fine-tuned jointly with the training of dialogue generation.…”

“…We also compare our approach with previous data augmentation or instance weighting methods: (i) CVAE-GAN(Li et al, 2019): a model that combines CVAE and GAN for augmenting the training data to generate more diversified expressions. (ii) Calibration(Shang et al, 2018): a calibration network measures the quality of data samples and enables weighted training for dialogue generation.…”

Modeling Personalization in Continuous Space for Response Generation via Augmented Wasserstein Autoencoders

“…Our model is motivated by following two points: 1) existing embedding based per-sona modeling methods cannot discover the common properties among users and train the embedding for different user independently, which cause (equal to learning) a very high-dimensional persona embedding and thus have a low data utilization efficiency or require a large amount of training data for each user. 2) benefited by the semantic capturing ability of WAEs, plenitude persona information can be gathered into the continuous space (Li et al, 2019). To this end, we build our model upon the state-of-the-art conversation model WAE (Gu et al, 2019) to model utterance-level and the user-level information.…”