A corpus-based approach to help-desk response generation

Abstract

We are developing a corpus-based approach for the prediction of help-desk responses from features in customers' emails, where responses are represented at two levels of granularity: document and sentence. We present an automatic evaluation of the responses generated by our system, as well as a manual one involving human judges. The automatic evaluation involves textual comparisons between generated responses and responses composed by the help-desk operators. The results show that both levels of granularity produce good responses, addressing inquiries of different kinds. The human-based evaluation measures response informativeness, and confirms our conclusion that both levels of granularity produce useful responses.

SAWDUST: a Semi-Automated Wizard Dialogue Utterance Selection Tool for domain-independent large-domain dialogue

“…In the selection approach, the response is formulated by simply picking the appropriate utterance from a set of previously observed utterances. This approach is used in many wizard of oz systems, where the wizard presses a button to select an utterance, as well as in many automated dialogue systems (Leuski et al, 2006;Zukerman and Marom, 2006;Sellberg and Jönsson, 2008) The resources required for the selection approach are a set of utterances to choose from and optionally, a set of pairs of context, response utterance to train automatic dialogue models. A wizard can generate such resources by performing two types of tasks.…”