Characterizing and recognizing spoken corrections in human-computer dialogue

Abstract

Miscommunication in speech recognition systems is unavoidable, but a detailed characterization of user corrections will enable speech systems to identify when a correction is taking place and to more accurately recognize the content of correction utterances. In this paper we investigate the adaptations of users when they encounter recognition errors in interactions with a voice-in/voice-out spoken language system. In analyzing more than 300 pairs of original and repeat correction utterances, matched on speaker and lexical content, we found overall increases in both utterance and pause duration from original to correction. Interestingly, corrections of misrecognition errors (CME) exhibited significantly heightened pitch variability, while corrections of rejection errors (CRE) showed only a small but significant decrease in pitch minimum. CME's demonstrated much greater increases in measures of duration and pitch variability than CRE's. These contrasts allow the development of decision trees which distinguish CME's from CRE's and from original inputs at 70-75% accuracy based on duration, pitch, and amplitude features.

Multimodal error correction for speech user interfaces

“…Previous studies have shown that accuracy significantly decreases in repeated spoken correction attempts [Levow 1998], and that users are more likely to switch modality following an error [Oviatt and VanGent 1996]. The present research extends these findings by demonstrating degraded recognition rates not only with respeaking but also with repeated spelled and handwritten corrections.…”

The dual of denial: Two uses of disconfirmations in dialogue and their prosodic correlates

“…Swerts and Ostendorf, 1997). This expectation is con®rmed by a bulk of recent work on hyperarticulate speech (e.g., Levow, 1998;Oviatt et al, 1998a,b;Soltau and Waibel, 1998;Erickson et al, 1998), a speaking style which can be seen both as the result of speech recognition errors and as an important source of such errors. Typically, hyperarticulate speech has an increased pitch and longer duration.…”

“…It can be used as a basis for choosing the veri®cation strategy employed by the system, but it may also be a cue to switch to a dierent recognition engine. Levow (1998) found that the probability of experiencing a recognition error after a correct recognition is 0.16, but immediately after an incorrect recognition it is 0.44. This increase is probably caused by the fact that the speakers used hyperarticulate speech when they noticed that the system had a problem recognizing their previous utterance.…”

Construction et ?volution des repr?sentations mentales lors des premières utilisations d' un service de dialogue en langage naturel

“…Par exemple, l'étude des messages correctifs : des invites à la correction implicite ou explicite[8], de la détection des messages correctifs à partir de la prosodie[9] 2 -Les serveurs DTMF (Dual Tone Modulation Frequency) dans lesquels l'utilisateur pilote le service via le clavier téléphonique.-Les serveurs vocaux à reconnaissance vocale de mots isolés ou enrobés qui nécessitent que l'utilisateur prononce des mots de commande. -Les agents intelligents dialoguants qui sont utilisés dans les systèmes non-arborescents permettant le langage naturel 3.…”