End-to-end joint learning of natural language understanding and dialogue manager

Yang, Xuesong; Chen, Yun-Nung; Hakkani‐Tür, Dilek; Crook, Paul A.; Li, Xiujun; Gao, Jianfeng; Deng, Li

doi:10.1109/icassp.2017.7953246

Cited by 70 publications

(55 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the development of a user simulator was complex and it took considerable time to built an appropriate user policy. Additionally, some studies [4,5,14,16] relied on considerable supervised data. Reference [16] proposed an end-to-end model by jointly training NLU and DM with supervised learning.…”

Section: Related Workmentioning

confidence: 99%

“…Additionally, some studies [4,5,14,16] relied on considerable supervised data. Reference [16] proposed an end-to-end model by jointly training NLU and DM with supervised learning. References [4,5,14] applied the demonstration data to speed up the convergence in a supervised manner.…”

Section: Related Workmentioning

confidence: 99%

“…Yang 2017 [16]: A end-to-end dialogue manager is implemented as those in Reference [16]. The hidden size of the LSTM for NLU and system action prediction are both set to 16.…”

Section: A Dialogue Manager Trained With Rule-s (Rule-m)mentioning

confidence: 99%

See 2 more Smart Citations

Cooperative Multi-Agent Reinforcement Learning with Conversation Knowledge for Dialogue Management

2020

View full text Add to dashboard Cite

Dialogue management plays a vital role in task-oriented dialogue systems, which has become an active area of research in recent years. Despite the promising results brought from deep reinforcement learning, most of the studies need to develop a manual user simulator additionally. To address the time-consuming development of simulator policy, we propose a multi-agent dialogue model where an end-to-end dialogue manager and a user simulator are optimized simultaneously. Different from prior work, we optimize the two-agents from scratch and apply the reward shaping technology based on adjacency pairs constraints in conversational analysis to speed up learning and to avoid the derivation from normal human-human conversation. In addition, we generalize the one-to-one learning strategy to one-to-many learning strategy, where a dialogue manager can be concurrently optimized with various user simulators, to improve the performance of trained dialogue manager. The experimental results show that one-to-one agents trained with adjacency pairs constraints can converge faster and avoid derivation. In cross-model evaluation with human users involved, the dialogue manager trained in one-to-many strategy achieves the best performance. from scratch without supervised initializing process. For user simulator reward function, we use the reward shaping technique [11] based on the adjacency pairs in conversational analysis [12] to make the simulator learn real user behaviors quickly. In addition, we generalize the one-to-one learning strategy to one-to-many learning strategy where a dialogue manager cooperates with various user simulators to improve the performance of trained dialogue manager. We obtain these various user simulators through changing the adjacency pairs settings, and then we mixture them with a dialogue manager to optimize the cooperative policies via multi-agent reinforcement learning.Compared with MADM without the constraints, MADM trained with adjacency pairs constraints can converge faster and avoid derivation from normal human-human conversation. The experimental results also show that the dialogue manager trained with one-to-many strategy achieves the best performance in cross-model evaluation with human users involved. To summary, our main contributions in this work are three-fold:

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Cooperative Multi-Agent Reinforcement Learning with Conversation Knowledge for Dialogue Management

2020

View full text Add to dashboard Cite

show abstract

“…Training probabilistic models to predict such intents has been aided in recent years by combinations of deep neural network models [2,6] and distributed word representations in the form of word embeddings [3,4]. Despite these improvements, building a system that performs adequately in a production role remains a substantial challenge, even when we focus on the NLU component only and leave aside the dialogue state management [9].…”

Section: Introductionmentioning

confidence: 99%

Enhancing Domain-Specific Supervised Natural Language Intent Classification with a Top-Down Selective Ensemble Model

Jenset¹,

McGillivray

2019

MAKE

View full text Add to dashboard Cite

Natural Language Understanding (NLU) systems are essential components in many industry conversational artificial intelligence applications. There are strong incentives to develop a good NLU capability in such systems, both to improve the user experience and in the case of regulated industries for compliance reasons. We report on a series of experiments comparing the effects of optimizing word embeddings versus implementing a multi-classifier ensemble approach and conclude that in our case, only the latter approach leads to significant improvements. The study provides a high-level primer for developing NLU systems in regulated domains, as well as providing a specific baseline accuracy for evaluating NLU systems for financial guidance.

show abstract

“…End-to-end learning overcomes these problems by training SDSs using the data obtained from the available dialogues. Endto-end learning has been applied to dialogue systems such as short response generation [1], open domain conversational dialogue systems [2], task-oriented dialogue systems [3], and the joint training of understanding models with a dialogue manager to predict system actions [4].…”

Section: Introductionmentioning

confidence: 99%

Towards End-to-End Spoken Dialogue Systems with Turn Embeddings

Bayer¹,

Stepanov²,

Riccardi³

2017

Interspeech 2017

View full text Add to dashboard Cite

Training task-oriented dialogue systems requires significant amount of manual effort and integration of many independently built components; moreover, the pipeline is prone to errorpropagation. End-to-end training has been proposed to overcome these problems by training the whole system over the utterances of both dialogue parties. In this paper we present an end-to-end spoken dialogue system architecture that is based on turn embeddings. Turn embeddings encode a robust representation of user turns with a local dialogue history and they are trained using sequence-to-sequence models. Turn embeddings are trained by generating the previous and the next turns of the dialogue and additionally perform spoken language understanding. The end-to-end spoken dialogue system is trained using the pre-trained turn embeddings in a stateful architecture that considers the whole dialogue history. We observe that the proposed spoken dialogue system architecture outperforms the models based on local-only dialogue history and it is robust to automatic speech recognition errors.

show abstract

End-to-end joint learning of natural language understanding and dialogue manager

Cited by 70 publications

References 27 publications

Cooperative Multi-Agent Reinforcement Learning with Conversation Knowledge for Dialogue Management

Cooperative Multi-Agent Reinforcement Learning with Conversation Knowledge for Dialogue Management

Enhancing Domain-Specific Supervised Natural Language Intent Classification with a Top-Down Selective Ensemble Model

Towards End-to-End Spoken Dialogue Systems with Turn Embeddings

Contact Info

Product

Resources

About