Task-Oriented Dialog Systems That Consider Multiple Appropriate Responses under the Same Context

Abstract

Conversations have an intrinsic one-to-many property, which means that multiple responses can be appropriate for the same dialog context. In task-oriented dialogs, this property leads to different valid dialog policies towards task completion. However, none of the existing task-oriented dialog generation approaches takes this property into account. We propose a Multi-Action Data Augmentation (MADA) framework to utilize the one-to-many property to generate diverse appropriate dialog responses. Specifically, we first use dialog states to summarize the dialog history, and then discover all possible mappings from every dialog state to its different valid system actions. During dialog system training, we enable the current dialog state to map to all valid system actions discovered in the previous process to create additional state-action pairs. By incorporating these additional pairs, the dialog policy learns a balanced action distribution, which further guides the dialog model to generate diverse responses. Experimental results show that the proposed framework consistently improves dialog policy diversity, and results in improved response diversity and appropriateness. Our model obtains state-of-the-art results on MultiWOZ.

ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems

“…ConvLab-2 extends Sequicity (Lei et al, 2018) to multi-domain scenarios: when the model senses that the current domain has switched, it resets the belief span, which records information of the current domain. ConvLab-2 also integrates DAMD (Zhang et al, 2019) which obtains state-of-the-art results on MultiWOZ. As for the DealOrNoDeal dataset, we provide the ROLL-OUTS RL policy proposed by Lewis et al (2017).…”

RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling

“…Task Definition: We recast dialogue response generation a sequence-to-sequence problem: encoding dialogue context to decode system response. Model: To this task, we use the Domain-Aware Multi-Decoder (DAMD) model (Zhang et al, 2019b) which achieves state-of-the-art performance on the MultiWOZ 2.0 dataset (Budzianowski et al, 2018). It's an end-to-end model proposed to handle the multi-domain response generation problem, which uses one encoder to encode dialogue context and three decoders to decode the belief span, system action and system response.…”

“…The inform rate measures the percentage that the output contains the appropriate entity the user asks for, and the success rate estimates the proportion that all the requested attributes have been answered. The combined score is calculated via (inf orm + success) * 0.5 + BLEU as an overall quality (Zhang et al, 2019b). Still, multi-domain dialogues exhibit a high difficulty level.…”

ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning

“…as a ranked list of plausible answers, as demonstrated in Figure 1. Such sets of diverse answers represent the nature of common sense knowledge and may be useful in applications such as dialogue systems, where multiple responses are appropriate for a given context (Zhang et al, 2019b).…”