Deep Reinforcement Learning for Dialogue Generation

Li, Jiwei; Monroe, Will; Ritter, Alan; Jurafsky, Dan; Galley, Michel; Gao, Jianfeng

doi:10.18653/v1/d16-1127

Cited by 912 publications

(726 citation statements)

References 39 publications

Supporting

Mentioning

692

Contrasting

Unclassified

Order By: Relevance

“…For task-specific dialogue Zhao and Eskenazi, 2016;Cuayáhuitl et al, 2016;Williams and Zweig, 2016;Li et al, 2017b,c;Peng et al, 2017), the reward function is usually based on task completion rate, and thus is easy to define. For the much harder problem of open-domain dialogue generation (Li et al, 2016e;Weston, 2016), hand-crafted reward functions are used to capture desirable conversation properties. Li et al (2016d) propose DRL-based diversitypromoting Beam Search (Koehn et al, 2003) for response generation.…”

Section: Related Work and Contributionsmentioning

confidence: 99%

Deep Active Learning for Dialogue Generation

Asghar

Poupart

Jiang

et al. 2017

Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017)

View full text Add to dashboard Cite

We propose an online, end-to-end, neural generative conversational model for opendomain dialogue. It is trained using a unique combination of offline two-phase supervised learning and online human-inthe-loop active learning. While most existing research proposes offline supervision or hand-crafted reward functions for online reinforcement, we devise a novel interactive learning mechanism based on hamming-diverse beam search for response generation and one-character userfeedback at each step. Experiments show that our model inherently promotes the generation of semantically relevant and interesting responses, and can be used to train agents with customized personas, moods and conversational styles.

show abstract

Section: Related Work and Contributionsmentioning

confidence: 99%

Deep Active Learning for Dialogue Generation

Asghar

Poupart

Jiang

et al. 2017

Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017)

View full text Add to dashboard Cite

show abstract

“…Recent years have seen neural networks being applied to all key parts of the typical modern IR pipeline, such core ranking algorithms [26,42,51], click models [9,10], knowledge graphs [8,35], text similarity [28,47], entity retrieval [52,53], language modeling [5], question answering [22,56], and dialogue systems [34,54].…”

Section: Motivationmentioning

confidence: 99%

“…Targeting this newly emerging demand, some models have been proposed to respond by generating natural language replies on the y, rather than by (re)ranking a xed set of items or extracting passages from existing pages. Examples are conversational and dialog systems [7,34,54] or machine reading and question answering tasks where the model either infers the answer from unstructured data, like textual documents that do not necessarily feature the answer literally [21,22,46,56], or generates natural language given structured data, like data from knowledge graphs or from external memories [1,18,33,37,40].…”

Section: Objectivesmentioning

confidence: 99%

Neural Networks for Information Retrieval

Kenter¹,

Borisov

Gysel

et al. 2018

Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

Machine learning plays a role in many aspects of modern IR systems, and deep learning is applied in all of them. The fast pace of modernday research has given rise to many di erent approaches for many di erent IR problems. The amount of information available can be overwhelming both for junior students and for experienced researchers looking for new research topics and directions. Additionally, it is interesting to see what key insights into IR problems the new technologies are able to give us. The aim of this full-day tutorial is to give a clear overview of current tried-and-trusted neural methods in IR and how they bene t IR research. It covers key architectures, as well as the most promising future directions. MOTIVATIONPrompted by the advances of deep learning in computer vision research, neural networks have resurfaced as a popular machine learning paradigm in many other directions of research as well, including information retrieval. Recent years have seen neural networks being applied to all key parts of the typical modern IR pipeline, such core ranking algorithms [26,42,51], click models [9,10], knowledge graphs [8,35], text similarity [28,47], entity retrieval [52,53], language modeling [5], question answering [22,56], and dialogue systems [34,54].A key advantage that sets neural networks apart from many learning strategies employed earlier, is their ability to work from raw input data. E.g., when given enough training data, well-designed networks can become feature extractors themselves, e.g., incorporating basic input characteristics such as term frequency (tf) and term saliency (idf)-that used to be pre-calculated o ine-in their initial layers. Where designing features used to be a crucial aspect and contribution of newly proposed IR approaches, the focus * Corresponding author.SIGIR '17, Shinjuku, Tokyo, Japan has shifted to designing network architectures instead. As a consequence, many di erent architectures and paradigms have been proposed, such as auto-encoders, recursive networks, recurrent networks, convolutional networks, various embedding methods, deep reinforcement and deep q-learning, and, more recently, generative adversarial networks, of which most have been applied in IR settings. The aim of the neural networks for IR (NN4IR) tutorial is to provide a clear overview of the main network architectures currently applied in IR and to show explicitly how they relate to previous work. The tutorial covers methods applied in industry and academia, with in-depth insights into the underlying theory, core IR tasks, applicability, key assets and handicaps, scalability concerns and practical tips and tricks.We expect the tutorial to be useful both for academic and industrial researchers and practitioners who either want to develop new neural models, use them in their own research in other areas or apply the models described here to improve actual IR systems. OBJECTIVESThe material in the tutorial covers a broad range of IR applications. It is structured as follows:Preliminaries (60 minutes). The rece...

show abstract

“…Reinforcement Learning can be used to improve dialogue managers, e.g. for transitions between dialogue states (Rieser and Lemon 2011), for nongoal-orientated dialogues (Li et al 2016), for botbot dialogues and for inventing new languages by agents (Das et al 2017). …”

Section: Deep Reinforcement Learningmentioning

confidence: 99%

AIA: Artificial intelligence for art

Lisek¹

2018

Electronic Workshops in Computing

View full text Add to dashboard Cite

There are limits to state-of-the-art AI that separate it from human-like intelligence. Today's AI algorithms are limited in how much previous knowledge they are able to keep through each new training phase and how much they can reuse. There is domain called AGI where will be possible to find solutions for this problems. Artificial general intelligence (AGI) describes research that aims to create machines capable of general intelligent action. "General" means that one AI program realises number of different tasks and the same code can be used in many applications.Artificial intelligence. Recurrent neural network. Reinforcement learning.

show abstract

Deep Reinforcement Learning for Dialogue Generation

Cited by 912 publications

References 39 publications

Deep Active Learning for Dialogue Generation

Deep Active Learning for Dialogue Generation

Neural Networks for Information Retrieval

AIA: Artificial intelligence for art

Contact Info

Product

Resources

About