Multi-turn Dialogue Response Generation in an Adversarial Learning Framework

Abstract

We propose an adversarial learning approach for generating multi-turn dialogue responses. Our proposed framework, hredGAN, is based on conditional generative adversarial networks (GANs). The GAN's generator is a modified hierarchical recurrent encoder-decoder network (HRED) and the discriminator is a word-level bidirectional RNN that shares context and word embeddings with the generator. During inference, noise samples conditioned on the dialogue history are used to perturb the generator's latent space to generate several possible responses. The final response is the one ranked best by the discriminator. The hredGAN shows improved performance over existing methods: (1) it generalizes better than networks trained using only the log-likelihood criterion, and (2) it generates longer, more informative and more diverse responses with high utterance and topic relevance even with limited training data. This improvement is demonstrated on the Movie triples and Ubuntu dialogue datasets using both automatic and human evaluations.

An Adversarial Learning Framework For A Persona-Based Multi-Turn Dialogue Model

“…where λ G and λ M are training hyperparamters and L cGAN (G, D) and L M LE (G) are defined in Eqs. (5) and (7) of Olabiyi et al (2018) respectively. Please note that the generator G and discriminator D share the same encoder and embedding representation of the word tokens.…”

“…The proposed architecture of phredGAN is very similar to that of hredGAN (Olabiyi et al, 2018). The only difference is that the dialogue history is now x i = (x 1 , c 1 ), (x 2 , c 2 ), · · · , (x i , c i ) where c i is additional input that represents the speaker and/or utterance attributes.…”

“…This forces the generator to draw a connection between the generated responses and the utterance attributes such as speaker identity. Noise Injection: As in Olabiyi et al (2018), we also explore different noise injection methods.…”

“…(3) where z i the noise sample and depends on the choice of either utterance-level or word-level noise input into the generator (Olabiyi et al, 2018).…”

“…Both systems aim to maintain the response quality of hredGAN and still capture speaker and other attribute modalities within the conversation. In fact, both systems use the same generator architecture (PHRED generator), i.e., an hredGAN generator (Olabiyi et al, 2018) with additional utterance attribute representation at its encoder and decoder inputs as depicted in Figure 1. Conditioning on external attributes can be seen as another input modality as is the utterance into the underlying system.…”