End-to-End Learning of Task-Oriented Dialogs

Abstract

In this thesis proposal, we address the limitations of conventional pipeline design of taskoriented dialog systems and propose end-toend learning solutions. We design neural network based dialog system that is able to robustly track dialog state, interface with knowledge bases, and incorporate structured query results into system responses to successfully complete task-oriented dialog. In learning such neural network based dialog systems, we propose hybrid offline training and online interactive learning methods. We introduce a multi-task learning method in pre-training the dialog agent in a supervised manner using task-oriented dialog corpora. The supervised training agent can further be improved via interacting with users and learning online from user demonstration and feedback with imitation and reinforcement learning. In addressing the sample efficiency issue with online policy learning, we further propose a method by combining the learning-from-user and learningfrom-simulation approaches to improve the online interactive learning efficiency.

Flexibly-Structured Model for Task-Oriented Dialogues

“…Both issues make these approaches less practical in real-world applications. Our work is also akin to modularly connected end-to-end trainable networks (Wen et al, 2017b,a;Liu and Lane, 2018;Li et al, 2018;Zhong et al, 2018). Wen et al (2017b) includes belief state tracking and has two phases in training: the first phase uses belief state supervision, and then the second phase uses response generation supervision.…”

Fuzzy Classification of Multi-intent Utterances

“…While open-ended dialog systems engage with the user in order to participate in a conversation [10], task oriented dialog systems focus on completing specific tasks enunciated by the user in the form of utterances, i.e. written or spoken natural language statements [20]. These utterances describe particular goals as enunciated by the speaker.…”

Collaborative Multi-Agent Dialogue Model Training Via Reinforcement Learning

“…Machine learning for conversational agents has seen great advances (e.g. Tur and Mori, 2011;Gao et al, 2019;Singh et al, 1999;Oh and Rudnicky, 2000;Zen et al, 2009;Reiter and Dale, 2000;Rieser and Lemon, 2010), especially when adopting deep learning models (Deng and Mesnil et al, 2015;Wen et al, 2015Wen et al, , 2017Papangelis et al, 2018;Liu and Lane, 2018b;Li et al, 2017;Williams et al, 2017;Liu and Lane, 2018a). Most of these works, however, suffer from the lack of data availability as it is very challenging to design sample-efficient learning algorithms for problems as complex as training agents capable of meaningful conversations.…”

“…Employing a user simulator is an established method for dialogue policy learning (Schatzmann et al, 2007, among others) and end-to-end dialogue training (Asri et al, 2016;Liu and Lane, 2018b). Training two conversational agents concurrently has been proposed by Georgila et al (2014); training them via natural language communication was partially realized by Liu and Lane (2017), as they train agents that receive text input but generate dialogue acts.…”