Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems

Wang, Jian; Liu, Junhao; Bi, Wei; Liu, Xiaojiang; He, Kejing; Xu, Ruifeng; Yang, Min

doi:10.18653/v1/2020.coling-main.362

Cited by 27 publications

(22 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the evolution of deep neural networks, several efforts have been made toward end-to-end task-oriented dialog systems [18], [19], [20]. Although these efforts have achieved compelling success, they focus on the pure textual modality, i.e., the single-modality task-oriented dialog system.…”

Section: Task-oriented Dialog Systemsmentioning

confidence: 99%

Multimodal Dialog Systems with Dual Knowledge-enhanced Generative Pretrained Language Model

Chen¹,

Song²,

Jing³

et al. 2022

Preprint

View full text Add to dashboard Cite

Text response generation for multimodal task-oriented dialog systems, which aims to generate the proper text response given the multimodal context, is an essential yet challenging task. Although existing efforts have achieved compelling success, they still suffer from two pivotal limitations: 1) overlook the benefit of generative pre-training, and 2) ignore the textual context related knowledge.To address these limitations, we propose a novel dual knowledge-enhanced generative pretrained language model for multimodal task-oriented dialog systems (DKMD), consisting of three key components: dual knowledge selection, dual knowledge-enhanced context learning, and knowledge-enhanced response generation. To be specific, the dual knowledge selection component aims to select the related knowledge according to both textual and visual modalities of the given context. Thereafter, the dual knowledge-enhanced context learning component targets seamlessly integrating the selected knowledge into the multimodal context learning from both global and local perspectives, where the cross-modal semantic relation is also explored. Moreover, the knowledge-enhanced response generation component comprises a revised BART decoder, where an additional dot-product knowledge-decoder attention sub-layer is introduced for explicitly utilizing the knowledge to advance the text response generation. Extensive experiments on a public dataset verify the superiority of the proposed DKMD over state-of-the-art competitors.

show abstract

Section: Task-oriented Dialog Systemsmentioning

confidence: 99%

Multimodal Dialog Systems with Dual Knowledge-enhanced Generative Pretrained Language Model

Chen¹,

Song²,

Jing³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…With the remarkable success of deep neural networks, several end-to-end task-oriented dialog systems have been proposed recently [16,33]. Particularly, some dialog systems consider domain knowledge to improve their performance [4,37], and some introduce deep reinforcement learning to strengthen the generative dialog systems [7,15,19].…”

Section: Related Work 21 Unimodal Dialog Systemsmentioning

confidence: 99%

Multimodal Dialog System: Relational Graph-based Context-aware Question Understanding

Zhang

Gao

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Multimodal dialog system has attracted increasing attention from both academia and industry over recent years. Although existing methods have achieved some progress, they are still confronted with challenges in the aspect of question understanding (i.e., user intention comprehension). In this paper, we present a relational graph-based context-aware question understanding scheme, which enhances the user intention comprehension from local to global. Specifically, we first utilize multiple attribute matrices as the guidance information to fully exploit the product-related keywords from each textual sentence, strengthening the local representation of user intentions. Afterwards, we design a sparse graph attention network to adaptively aggregate effective context information for each utterance, completely understanding the user intentions from a global perspective. Moreover, extensive experiments over a benchmark dataset show the superiority of our model compared with several state-of-the-art baselines. CCS CONCEPTS• Computing methodologies → Discourse, dialogue and pragmatics.

show abstract

“…There are also several works applying separate memories to model dialog history and KB information so as to further enhance the performance of TDSs (Raghu, Gupta et al 2019;Reddy et al 2019;Chen, Xu, and Xu 2019;Wang et al 2020;He et al 2021). For example, multi-level memory (Reddy et al 2019) utilized a multi-level memory to model the KB results, rather than using the form of triples.…”

Section: Task-oriented Dialog Systemsmentioning

confidence: 99%

“…WMM2Seq (Chen, Xu, and Xu 2019) introduced a working memory to coordinate two separated memories. DDMN (Wang et al 2020) employed a dual dynamic memory network to model the dialog context and KB information. However, previous models do not explicitly explore the auxiliary reasoning tasks to help model the dialog context and KB information.…”

Section: Task-oriented Dialog Systemsmentioning

confidence: 99%

“…Second, previous study (Carbonell 1983) argues that the users of a dialog system are prone to utilize succinct language and often drop entities appeared in historical utterances. Nevertheless, the Seq2Seq model tends to neglect how the conversation evolves as information progresses and thus leads to incoherent and ungrammatical responses that are dominated by words appearing with high frequency in the training data (Wang et al 2020). As shown in Table 1, it is difficult for the Seq2Seq model to infer the address of gas station from a long concatenated dialog context when answering the second user question.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exploring Auxiliary Reasoning Tasks for Task-oriented Dialog Systems with Meta Cooperative Learning

Qin

Yang

Jiang

et al. 2021

AAAI

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a Meta Cooperative Learning (MCL) framework for task-oriented dialog systems (TDSs). Our model consists of an auxiliary KB reasoning task for learning meta KB knowledge, an auxiliary dialogue reasoning task for learning dialogue patterns, and a TDS task (primary task) that aims at not only retrieving accurate entities from KB but also generating natural responses, which are coordinated to achieve collective success in both retrieving accurate KB entities and generating human-like responses via meta learning. Concretely, the dialog generation model amalgamates complementary meta KB and dialog knowledge from two novel auxiliary reasoning tasks that together provide integrated guidance to build a high-quality TDS by adding regularization terms to force primary network to produce similar results to auxiliary networks. While MCL automatically learns appropriate labels for the two auxiliary reasoning tasks from the primary task, without requiring access to any further data. The key idea behind MCL is to use the performance of the primary task, which is trained alongside the auxiliary tasks in one iteration, to improve the auxiliary labels for the next iteration with meta learning. Experimental results on three benchmark datasets show that MCL can generate higher quality responses compared to several strong baselines in terms of both automatic and human evaluations. Code to reproduce the results in this paper is available at: https://github.com/siat-nlp/MCL.

show abstract

Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems

Cited by 27 publications

References 20 publications

Multimodal Dialog Systems with Dual Knowledge-enhanced Generative Pretrained Language Model

Multimodal Dialog Systems with Dual Knowledge-enhanced Generative Pretrained Language Model

Multimodal Dialog System: Relational Graph-based Context-aware Question Understanding

Exploring Auxiliary Reasoning Tasks for Task-oriented Dialog Systems with Meta Cooperative Learning

Contact Info

Product

Resources

About