Neural Response Generation with Meta-words

Abstract

We present open domain response generation with meta-words. A meta-word is a structured record that describes various attributes of a response, and thus allows us to explicitly model the one-to-many relationship within open domain dialogues and perform response generation in an explainable and controllable manner. To incorporate meta-words into generation, we enhance the sequence-to-sequence architecture with a goal tracking memory network that formalizes meta-word expression as a goal and manages the generation process to achieve the goal with a state memory panel and a state controller. Experimental results on two large-scale datasets indicate that our model can significantly outperform several state-ofthe-art generation models in terms of response relevance, response diversity, accuracy of oneto-many modeling, accuracy of meta-word expression, and human evaluation. * Corresponding author. Message: last week I have a nice trip to New York! Meta-word: Act: yes-no question | Len: 8 | Copy: true | Utts: false | Spe: medium Response 1: Is New York more expensive than California? Meta-word: Act: wh-question | Len: 17 | Copy: false | Utts: true | Spe: high Response 2: Cool, sounds great! What is the tallest building in this city, Chrysler building? Meta-word: Act: statement | Len: 13 | Copy: false | Utts: true | Spe: low Response 3: I don't know what you are talking about. But it seems good.

Diverse and Informative Dialogue Generation with Context-Specific Commonsense Knowledge Awareness

“…Introducing additional contents into the dialogue generation is also helpful. (Xu et al, 2019) uses meta-words; (Zhu et al, 2019) uses the retrieved existing dialogues. However, the leading cause of generating generic responses is that the model can not obtain enough background knowledge from the query message (Ghazvininejad et al, 2018;.…”

“…Such low-quality responses always reduce the attractiveness of generative dialogue systems to end-users. Researchers have tried to tackle it from multiple aspects; for example, using the enhanced objective function (Li et al, 2016a); introducing additional contents (Xu et al, 2019). However, these methods haven't solved the issue thoroughly.…”

StyleDGPT: Stylized Response Generation with Pre-trained Language Models

“…Inspired by neural machine translation, early works apply the sequence-to-sequence model to this task and achieve promising results (Ritter et al, 2011;Shang et al, 2015;Vinyals and Le, 2015). Since then, various architectures have been proposed to address the key challenges in open-domain dialogue systems, including suppressing the generic responses (Li et al, 2015;Zhao et al, 2017;Xing et al, 2017a), context modeling (Serban et al, 2016Xing et al, 2017b;Zhang et al, 2019a), controlling the attributes of responses (Xu et al, 2019;Zhou et al, 2017;Zhang et al, 2018a;Wang et al, 2018;See et al, 2019) and incorporating different types knowledge into generation (Li et al, 2016;Zhang et al, 2018b;Zhou et al, 2017;Zhao et al, 2020). In this work, we study the problem of stylized response generation, which aims to incorporate the style information from non-parallel data into the generation process.…”

Consistent Response Generation with Controlled Specificity

“…Controlling the properties of generated responses is also related to our study. Xu et al (2019) and Ko et al (2019) allow for the control of dialogue-acts, length, and specificity of responses; however, they are resource intensive and thus require an external annotated corpus. In contrast, SC-Seq2Seq (Zhang et al, 2018a) achieves control of response specificity without dependence on external resources, which is most relevant to our study.…”