Improving Open-Domain Dialogue Systems via Multi-Turn Incomplete Utterance Restoration

Abstract

In multi-turn dialogue, utterances do not always take the full form of sentences. These incomplete utterances will greatly reduce the performance of open-domain dialogue systems. Restoring more incomplete utterances from context could potentially help the systems generate more relevant responses. To facilitate the study of incomplete utterance restoration for open-domain dialogue systems, a large-scale multi-turn dataset Restoration-200K 1 is collected and manually labeled with the explicit relation between an utterance and its context. We also propose a "pick-andcombine" model to restore the incomplete utterance from its context. Experimental results demonstrate that the annotated dataset and the proposed approach significantly boost the response quality of both single-turn and multiturn dialogue systems.

Incomplete Utterance Rewriting as Semantic Segmentation

“…(iii) EM stands for the exact match ac- ) 73.2 64.6 68.6 59.5 53.0 56.0 50.7 45.1 47.7 92.3 89.6 92.4 85.1 Table 2: The experimental results of (Top) general and (Bottom) BERT-based results on MULTI. †: Results from Pan et al (2019). A bolded number in a column indicates a statistically significant improvement against all the baselines (p < 0.05), whereas underline numbers show comparable performances.…”

“…(ii) Transformer-based models consist of the basic transformer model (T-Gen) (Vaswani et al, 2017), the transformer-based pointer network (T-Ptr), and the transformer-based pointer generator (T-Ptr-Gen). (iii) State-of-theart models consist of Syntactic (Kumar and Joshi, 2016), PAC (Pan et al, 2019), GECOR (Quan et al, 2019), L-Ptr-λ and T-Ptr-λ (Su et al, 2019). We refer readers to their papers for more details.…”

“…Evaluation We employ both automatic metrics and human evaluations to evaluate our approach. As in literature (Pan et al, 2019), we examine RUN using the widely used automatic metrics BLEU, ROUGE, EM and Rewriting F-score. (i) BLEU n (B n ) evaluates how similar the rewritten utterances are to the golden ones via the cumulative n-gram BLEU score (Papineni et al, 2002).…”

“…2019a; Pan et al, 2019;Elgohary et al, 2019;. IUR aims to rewrite an incomplete utterance into an utterance which is semantically equivalent but self-contained to be understood without context.…”

“…Liu et al (2019a) imposed an intermediate structure span and decomposed the incomplete utterance rewriting into two sub-tasks. In dialogue generation, Pan et al (2019) presented a cascaded model which first picks words from the context via BERT, and then combines these words to generate the rewritten utterance, and Su et al (2019) distinguished the weights of context utterances and the incomplete utterance using a hyper-parameter λ.…”