A methodology for diagnostic evaluation of spoken human — machine dialogue

Dybkjær, Laila; Bernsen, Niels Ole; Dybkjær, Hans

doi:10.1006/ijhc.1997.0183

Cited by 8 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of course, we have only looked at communication problems due to speech recognition errors or incorrect default assumptions. While these errors are certainly the most frequently encountered in practical spoken dialogue systems (see e.g., Oviatt et al, 1998, they are not the only source of communication problems (see, e.g., Dybkjaer et al, 1998). What speech recognition errors and incorrect default assumptions have in common is that they result in a state where the system's beliefs are inconsistent with the user's intentions (the user intends to go to Reuver while the system appears to 'believe' that he/she wants to go to Utrecht CS).…”

Section: Discussionmentioning

confidence: 99%

“…Conversely, in some cases, the recognizer might tolerate misrecognitions without causing communication errors, i.e., when the recognized string is conceptually identical to the intended meaning of the user's utterance (e.g., "yeah" vs. "yes")(Taylor, King, Isard and Wright, 1998).2 For a more exhaustive analysis of potential sources of errors, seeDybkjaer, Bernsen and Dybkjaer (1998), who report on a Wizard of Oz experiment. See also section 7.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Untitled

Krahmer

Swerts

Theune

et al. 2001

International Journal of Speech Technology

View full text Add to dashboard Cite

Given the state of the art of current language and speech technology, errors are unavoidable in present-day spoken dialogue systems. Therefore, one of the main concerns in dialogue design is how to decide whether or not the system has understood the user correctly. In human-human communication, dialogue participants are continuously sending and receiving signals on the status of the information being exchanged. We claim that if spoken dialogue systems were able to detect such cues and change their strategy accordingly, the interaction between user and system would improve. The goals of the present study are therefore twofold: (i ) to find out which positive and negative cues people actually use in human-machine interaction in response to explicit and implicit verification questions and how informative these signals are, and (ii ) to explore the possibilities of spotting errors automatically and on-line. To reach these goals, we first perform a descriptive analysis, followed by experiments with memory-based machine learning techniques. It appears that people systematically use negative/marked cues when there are communication problems. The experiments using memory-based machine learning techniques suggest that it may be possible to spot errors automatically and on-line with high accuracy, in particular when focussing on combinations of cues. This kind of information may turn out to be highly relevant for spoken dialogue systems, e.g., by providing quantitative criteria for changing the dialogue strategy or speech recognition engine.

show abstract

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

Untitled

Krahmer

Swerts

Theune

et al. 2001

International Journal of Speech Technology

View full text Add to dashboard Cite

show abstract

“…Building on results from the Danish dialogue project (Dybkjaer et al, 1998b) and DISC, Dybkjaer and Bernsen (2000) discuss existing knowledge of SLDS usability evaluation. They claim that the general design goal of creating usable walk-up-and-use, shared-goal SLDSs may be systematically pursued by addressing 13 key usability issues.…”

Section: Usability Guidelinesmentioning

confidence: 99%