Learning to Select Knowledge for Response Generation in Dialog Systems

Abstract

End-to-end neural models for intelligent dialogue systems suffer from the problem of generating uninformative responses. Various methods were proposed to generate more informative responses by leveraging external knowledge. However, few previous work has focused on selecting appropriate knowledge in the learning process. The inappropriate selection of knowledge could prohibit the model from learning to make full use of the knowledge. Motivated by this, we propose an end-to-end neural model which employs a novel knowledge selection mechanism where both prior and posterior distributions over knowledge are used to facilitate knowledge selection. Specifically, a posterior distribution over knowledge is inferred from both utterances and responses, and it ensures the appropriate selection of knowledge during the training process. Meanwhile, a prior distribution, which is inferred from utterances only, is used to approximate the posterior distribution so that appropriate knowledge can be selected even without responses during the inference process. Compared with the previous work, our model can better incorporate appropriate knowledge in response generation. Experiments on both automatic and human evaluation verify the superiority of our model over previous baselines.

Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems

“…As the knowledge sometimes contains a lot of redundant entries in knowledge-grounded conversation, Lian et at. [17] propose a model with the knowledge selection mechanism which leverages both prior and posterior distributions over the knowledge to facilitate knowledge selection. Kim et al [12] propose a sequential knowledge transformer that employs a sequential latent variable model to better leverage the response information for the proper choice of the knowledge collection in multi-turn dialogue.…”

Know More about Each Other: Evolving Dialogue Strategy via Compound Assessment

“…• The KG-Net (Lian et al, 2019) makes use of posterior knowledge distribution in the training process for accurate informative response generation and achieves the state-of-the-art results on PersonaChat. In our strategic knowledge interaction, the parameters of knowledge encoder, utterance encoder and decoder were pre-trained with supervised learning.…”

“…CCM (Zhou et al, 2018) relies on structured knowledge to generate rich-information response. In Lian et al (2019), the posterior distribution is estimated and accurate knowledge is selected to boost informative generation. However, without thorough consideration and control on the knowledge utilization in multi-turn conversations, the above approaches are prone to produce repetitive and incoherent utterances.…”

Exploiting Text Matching Techniques for Knowledge-Grounded Conversation

“…For example, selecting Knowledge-3 will result in a response less coherent than the one based on Knowledge-1; Knowledge-3 mentions about the diversity of population in the U.S., rather than a fact about "New York." Despite the importance of this selection, only few studies [4], [14] have investigated this issue. Dinan et al [14] and Lian et al [4] proposed models that focus on choosing the correct knowledge sentence.…”

“…Despite the importance of this selection, only few studies [4], [14] have investigated this issue. Dinan et al [14] and Lian et al [4] proposed models that focus on choosing the correct knowledge sentence. However, they do not focus on how to model the context in a multi-turn dialogue setting, where they merely model a multi-turn dialogue as a single document.…”