Response Timing Detection Using Prosodic and Linguistic Information for Human-friendly Spoken Dialog Systems

Abstract

SummaryIf a dialog system can respond to the user as reasonably as a human, the interaction will become smoother. Timing of the response such as back-channels and turn-taking plays an important role in such a smooth dialog as in human-human interaction. We developed a response timing generator for such a dialog system. This generator uses a decision tree to detect the timing based on the features coming from some prosodic and linguistic information. The timing generator decides the action of the system at every 100 ms during the user's pause. In this paper, we describe a robust spoken dialog system using the timing generator. Subjective evaluation proved that almost all of the subjects experienced a friendly feeling from the system.

Backchannel Prediction for Mandarin Human-Computer Interaction

“…Kitaoka et al used first-order regression coefficients of pitch and power contours to describe patterns and generate response timing [29]. Nishimura et al pointed out that both the last short regions and the longer ones contained information which triggered backchannel responses [30].…”

Posteriori Restoration of Turn-Taking and ASR Results for Incorrectly Segmented Utterances

“…† † http://www.cs.waikato.ac.nz/ml/weka/ (14)- (18) We exploited five features that may be effective for our task by referring to previous work on turn-taking decision [10]- [12]. There might be other features that are effective, but exploring such features is among the future work.…”

“…Ohsuga et al identified prosodic features that are helpful for determining ends of turns with decision tree learning on the Japanese Map Task Corpus [11]. Kitaoka et al also used both prosodic and linguistic information to determine timing of system response generation [12]. Edlund et al developed a prosodic analysis tool to augment end-point detection [13].…”

An analysis of switching pause duration as a paralinguistic feature in expressive dialogues

“…Switching pauses [4], which are defined as pauses between turns, have been regarded as a distinctive property of spoken dialogue as a form of social interaction [5,6]. Unlike pauses in monologues or intrapersonal pauses in dialogues, the duration of switching pauses has an aspect similar to that of reaction time.…”

“…Although some previous studies [11,12] reported the effects of emotional state on the duration of intrautterance pauses, they did not deal with switching pauses. Furthermore, because the generation of response timing has been treated as an independent module from the speech synthesizer in most spoken dialogue systems [6,13], the finely tuned modeling of switching pause duration taking paralinguistic effects into account has specific importance in the design of responsive human interfaces.…”