Know More about Each Other: Evolving Dialogue Strategy via Compound Assessment

Abstract

In this paper, a novel Generation-Evaluation framework is developed for multi-turn conversations with the objective of letting both participants know more about each other. For the sake of rational knowledge utilization and coherent conversation flow, a dialogue strategy which controls knowledge selection is instantiated and continuously adapted via reinforcement learning. Under the deployed strategy, knowledge grounded conversations are conducted with two dialogue agents. The generated dialogues are comprehensively evaluated on aspects like informativeness and coherence, which are aligned with our objective and human instinct. These assessments are integrated as a compound reward to guide the evolution of dialogue strategy via policy gradient. Comprehensive experiments have been carried out on the publicly available dataset, demonstrating that the proposed method outperforms the other state-of-the-art approaches significantly.

PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable

“…It is worth noting that the above fine-tuning and inference procedures are set up for the dialogue generation without any specific objectives. If there exists a specific objective within the conversation, such as letting both participants know more about each other (Bao et al, 2019), the fine-tuning can proceed to maximize the pre-defined rewards with reinforcement learning (RL). Under such circumstances, our latent discrete variable can be naturally treated as action within RL, and thus the response selection can be straightforwardly solved by selecting the action that results in the maximum reward.…”

PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning

“…Self-chats have been widely used in the evaluation of dialogue systems (Li et al, 2016b;Bao et al, 2019;Roller et al, 2021), where a model plays the role of both partners in the conversation. As compared with human-bot conversations, self-chat logs can be collected efficiently at a cheaper price.…”

Conversational Graph Grounded Policy Learning for Open-Domain Conversation Generation

“…In this work, we investigate how to use prior dialog-transition information to facilitate dialog policy learning. Knowledge aware conversation generation There are growing interests in leveraging knowledge bases for generation of more informative responses (Dinan et al, 2019;Ghazvininejad et al, 2018;Moghe et al, 2018;Zhou et al, 2018;Liu et al, 2019;Bao et al, 2019;Xu et al, 2020). In this work, we employ a dialog-modeling oriented graph built from dialog corpora, instead of a external knowledge base, in order to facilitate multi-turn policy learning, instead of dialog informativeness improvement.…”