Pursuing and demonstrating understanding in dialogue

DeVault, David; Stone, Matthew

doi:10.1017/cbo9780511844492.003

Cited by 1 publication

(1 citation statement)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the system generates tree vs. man) due to perceptual uncertainty [7]. Thus, it has been argued, that dialogue systems interacting with users in real-world environments need principled communicative mechanisms for dealing with uncertainties, perceptual mismatches, and potential misunderstanding [14,15,16,17]. In this study, we look at (potentially less disturbing) defective color terms, which turn out to be hard to predict for objects in real-world images as well [6].…”

Section: Introductionmentioning

confidence: 99%

Do Hesitations Facilitate Processing of Partially Defective System Utterances? An Exploratory Eye Tracking Study

Haake¹,

Schimke²,

Betz

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

Spoken dialogue systems are predominantly evaluated using offline methods such as user ratings or task-oriented measures. Various phenomena in conversational speech, however, are known to affect the way the listener's comprehension unfolds over time, and not necessarily the final result of the comprehension process. For instance, in human reference comprehension, conversational signals like hesitations have been shown to ease processing of expressions referring to difficult-to-describe targets, as can primarily be observed in listeners' anticipatory eye movements rather than in their final reference resolution decision. In this study, we explore eye tracking for testing conversational dialogue systems, looking at how listeners process automatically generated referring expressions containing defective attributes. We investigate whether hesitations facilitate the processing of partially defective system utterances and track the user's eye movements when listening to expressions with: (i) semantically defective but fluently synthesized adjectives, (ii) defective and lengthened adjectives, i.e. containing a conversational uncertainty signal. Our results are encouraging: whereas the offline measure of task success does not show any differences between the two conditions, the listeners' eye movements suggest that processing of partially defective utterances might be facilitated by conversational hesitations.

show abstract