"So, which one is it?" The effect of alternative incremental architectures in a high-performance game-playing agent

Abstract

This paper introduces Eve, a highperformance agent that plays a fast-paced image matching game in a spoken dialogue with a human partner. The agent can be optimized and operated in three different modes of incremental speech processing that optionally include incremental speech recognition, language understanding, and dialogue policies. We present our framework for training and evaluating the agent's dialogue policies. In a user study involving 125 human participants, we evaluate three incremental architectures against each other and also compare their performance to human-human gameplay. Our study reveals that the most fully incremental agent achieves game scores that are comparable to those achieved in human-human gameplay, are higher than those achieved by partially and nonincremental versions, and are accompanied by improved user perceptions of efficiency, understanding of speech, and naturalness of interaction.

Conversational Image Editing: Incremental Intent Identification in a New Dialogue Task

“…They used images from MS COCO and CNNs for image recognition. Paetzel et al (2015) built an incremental dialogue system called "Eve", which could guess the correct image, out of a set of possible candidates, based on descriptions given by a human. The system was shown to perform nearly as well as humans.…”

Toward incremental dialogue act segmentation in fast-paced interactive dialogue systems

“…Briefly, this version of Eve includes the same incremental ASR used in our new DA segmentation pipeline (Plátek and Jurčíček, 2014), incremental language understanding to identify the target image (Naive Bayes classification), and an incremental dialogue policy that uses parameterized rules. See Paetzel et al (2015) for full details.…”

“…The current baseline agent (Paetzel et al, 2015) can only generate As-I and As-S dialogue acts. In future work, the fully automated pipeline presented here will enable us to expand the agent's dialogue policies to support additional patterns of interaction beyond its current skillset.…”

“…In this paper we explore the feasibility of incorporating an incremental dialogue act segmentation capability into an implemented, high-performance spoken dialogue agent that plays a time-constrained image-matching game with its users (Paetzel et al, 2015). This work is part of a longer-term research program that aims to use incremental (word-byword) language processing techniques to enable dialogue agents to support efficient, fast-paced interactions with a natural conversational style (DeVault et al, 2011;Ward and DeVault, 2015;Paetzel et al, 2015).…”

“…This work is part of a longer-term research program that aims to use incremental (word-byword) language processing techniques to enable dialogue agents to support efficient, fast-paced interactions with a natural conversational style (DeVault et al, 2011;Ward and DeVault, 2015;Paetzel et al, 2015).…”