Multi-turn Dialogue Response Generation in an Adversarial Learning Framework

Olabiyi, O.; Salimov, Alan; Khazane, Anish; Mueller, Erik T.

doi:10.48550/arxiv.1805.11752

Cited by 6 publications

(40 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [157], also the authors based on MC-GAN, introduced a model to produce high resolution images by adding the StackGAN to generator. Olabiyi et al [158] proposed an adversarial network to generate multi-turn dialogue answers. The framework is based on conditional GANs.…”

Section: Multi-stage Methodsmentioning

confidence: 99%

Image Synthesis with Adversarial Networks: a Comprehensive Survey and Case Studies

Shamsolmoali¹,

Zareapoor²,

Granger³

et al. 2020

Preprint

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have been extremely successful in various application domains such as computer vision, medicine, and natural language processing. Moreover, transforming an object or person to a desired shape become a well-studied research in the GANs. GANs are powerful models for learning complex distributions to synthesize semantically meaningful samples. However, there is a lack of comprehensive review in this field, especially lack of a collection of GANs loss-variant, evaluation metrics, remedies for diverse image generation, and stable training. Given the current fast GANs development, in this survey, we provide a comprehensive review of adversarial models for image synthesis. We summarize the synthetic image generation methods, and discuss the categories including image-to-image translation, fusion image generation, label-to-image mapping, and text-to-image translation. We organize the literature based on their base models, developed ideas related to architectures, constraints, loss functions, evaluation metrics, and training datasets. We present milestones of adversarial models, review an extensive selection of previous works in various categories, and present insights on the development route from the model-based to data-driven methods. Further, we highlight a range of potential future research directions. One of the unique features of this review is that all software implementations of these GAN methods and datasets have been collected and made available in one place at https://github.com/pshams55/GAN-Case-Study.

show abstract

Section: Multi-stage Methodsmentioning

confidence: 99%

Image Synthesis with Adversarial Networks: a Comprehensive Survey and Case Studies

Shamsolmoali¹,

Zareapoor²,

Granger³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, several generative model-based methods have been proposed. The Generative Adversarial Network (GAN) was proposed in image generation (Goodfellow et al 2014) and applied to text generation (Yu et al 2017) and dialogue generation (Li et al 2017;Xu et al 2018;Olabiyi et al 2018). Currently, training with GAN for dialogue generation is very unstable and requires pre-training.…”

Section: Related Workmentioning

confidence: 99%

“…BLEU BLEU-n calculates the percentage of n-gram matching between all of the generated sentences and all of the reference sentences (Papineni et al 2002). We calculated the corpus-level BLEU-1 and BLEU-2 scores that measure the degree of unigram and up to bigram matching.…”

Section: Evaluation Detailsmentioning

confidence: 99%

“…Some studies have artificially promoted diversity. A diversity-promoting objective function based on Maximum Mutual Information (MMI) first addressed this kind of problem (Li et al 2016a), and various generative model-based methods (e.g., GAN and VAE) have been proposed (Li et al 2017;Xu et al 2018;Olabiyi et al 2018;Cao and Clark 2017;Zhao, Zhao, and Eskenazi 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Another Diversity-Promoting Objective Function for Neural Dialogue Generation

Nakamura¹,

Sudoh²,

Yoshino³

et al. 2018

Preprint

View full text Add to dashboard Cite

Although generation-based dialogue systems have been widely researched, the response generations by most existing systems have very low diversities. The most likely reason for this problem is Maximum Likelihood Estimation (MLE) with Softmax Cross-Entropy (SCE) loss. MLE trains models to generate the most frequent responses from enormous generation candidates, although in actual dialogues there are various responses based on the context. In this paper, we propose a new objective function called Inverse Token Frequency (ITF) loss, which individually scales smaller loss for frequent token classes and larger loss for rare token classes. This function encourages the model to generate rare tokens rather than frequent tokens. It does not complicate the model and its training is stable because we only replace the objective function. On the OpenSubtitles dialogue dataset, our loss model establishes a state-of-the-art DIST-1 of 7.56, which is the unigram diversity score, while maintaining a good BLEU-1 score. On a Japanese Twitter replies dataset, our loss model achieves a DIST-1 score comparable to the ground truth.

show abstract

“…Existing dialogue systems are broken into two major categories, open-domain dialogue systems, which focus on non-task related conversations, and task-oriented dialogue systems, which focus on user task completion. A typical open-domain system uses an end-to-end neural architecture often trained with input Preprint and output utterances from human-to-human conversations (Sutskever, Vinyals, and Le 2014;Serban et al 2016Serban et al , 2017Olabiyi et al 2018Zhang et al 2019). While open-domain systems are optimized for engaging in human-like conversation, they lack any inherent ability to interface with any other systems on behalf of their conversation partner.…”

Section: Introductionmentioning

confidence: 99%

DLGNet-Task: An End-to-end Neural Network Framework for Modeling Multi-turn Multi-domain Task-Oriented Dialogue

Olabiyi,

Bhattarai,

Bruss

et al. 2020

Preprint

View full text Add to dashboard Cite

Task oriented dialogue (TOD) requires the complex interleaving of a number of individually controllable components with strong guarantees for explainability and verifiability. This has made it difficult to adopt the multi-turn multi-domain dialogue generation capabilities of streamlined end-to-end opendomain dialogue systems. In this paper, we present a new framework, DLGNet-Task, a unified task-oriented dialogue system which employs autoregressive transformer networks such as DLGNet and GPT-2/3 to complete user tasks in multiturn multi-domain conversations. Our framework enjoys the controllable, verifiable, and explainable outputs of modular approaches, and the low development, deployment and maintenance cost of end-to-end systems. Treating open-domain system components as additional TOD system modules allows DLGNet-Task to learn the joint distribution of the inputs and outputs of all the functional blocks of existing modular approaches such as, natural language understanding (NLU), state tracking, action policy, as well as natural language generation (NLG). Rather than training the modules individually, as is common in real-world systems, we trained them jointly with appropriate module separations. When evaluated on the MultiWOZ2.1 dataset, DLGNet-Task shows comparable performance to the existing state-of-the-art approaches. Furthermore, using DLGNet-Task in conversational AI systems reduces the level of effort required for developing, deploying, and maintaining intelligent assistants at scale.

show abstract

Multi-turn Dialogue Response Generation in an Adversarial Learning Framework

Cited by 6 publications

References 0 publications

Image Synthesis with Adversarial Networks: a Comprehensive Survey and Case Studies

Image Synthesis with Adversarial Networks: a Comprehensive Survey and Case Studies

Another Diversity-Promoting Objective Function for Neural Dialogue Generation

DLGNet-Task: An End-to-end Neural Network Framework for Modeling Multi-turn Multi-domain Task-Oriented Dialogue

Contact Info

Product

Resources

About