Usability issues in spoken dialogue systems

Abstract

Whilst Spoken Language Dialogue Systems (SLDSs) technology has made good progress in recent years, the issue of SLDS usability is still lagging behind both theoretically and in actual SLDS development and evaluation. However, as more products reach the market and competition intensifies, there is growing recognition of the importance of systematically understanding the factors which must be taken into account in order to optimise SLDS usability. Ideally, this understanding should be comprehensive (i.e. include all major human factors perspectives on SLDSs), and exhaustive (i.e. describe each perspective as it pertains to the detailed development and evaluation of any possible SLDS). This paper addresses the requirement of comprehensiveness by decomposing the complex space of SLDS usability best practice into eleven issues which should be considered by developers during specification, design, development and evaluation. The discussion of each issue is aimed to support the developer in building SLDSs which are likely to generate user satisfaction, which are perceived to be easy to understand and control, and which enable smooth user-system interaction. Based on the best practice issues discussed, criteria for evaluating SLDS usability are proposed. Several limits to our current understanding of SLDS usability are highlighted. S1: Hello. You are connected to a spoken dialogue system prototype. This is the automatic train timetable information. You may inquire about train connections to one thousand German stations. From where to where would you like to go? U1: From Siegen to Cologne. S2: When would you like to go from Siegen to Cologne? U2: On 29th June. S3: At which time would you like to leave on Saturday 29th June? U3: At nine o'clock. S4: So you would like to leave at nine o'clock in the morning? U4: Yes. S5: There is the following connection with express train 3312 from Siegen at 9:09, arrival in Cologne at 10:39. Would you like to have the connection repeated? U5: No. S6: Would you like another connection? U6: No. S7: Then I wish you a pleasant trip.

Explicit feedback from users attenuates memory biases in human-system dialogue

“…The main advantage associated with the user or the system repeating the information presented by the other partner is that it allows this partner to check that the information presented was understood correctly (Cahn & Brennan, 1999;Dybkjaer & Bernsen, 2001;Dybkjaer & Bernsen, 2000). Furthermore, system explicit acceptance can increase user satisfaction, especially when the information repeated is important within the context of the task framework (Stent, Dowding, Gawron, Bratt, & Moore, 1999).…”

A statistical approach to spoken dialog systems design and evaluation

“…In the same line, the DISC 4 project (Spoken Language Dialogue Systems and Components) (Failenschmid et al, 1999) proposes different measures and criteria to be considered in the evaluation. Finally, a set of 15 criteria to evaluate the system usability can be found in (Dybkjaer and Bernsen, 2000).…”

Spoken Dialogue Interfaces: Integrating Usability

“…At the end of the implementation, pre-final versions of the system should be tested by potential users in order to evaluate the usability. For spoken dialogue interfaces, a set of 15 objective (quantitative or qualitative) and subjective usability evaluation criteria have been proposed [7], including modality appropriateness, input recognition adequacy, naturalness, output voice quality, output phrasing adequacy, feedback adequacy, adequacy of dialogue initiative relative to the task(s), naturalness of the dialogue structure relative to the task(s), sufficiency of task and domain coverage, sufficiency of the system's reasoning capabilities, sufficiency of interaction guidance, error handling adequacy, sufficiency of adaptation to user differences, number of interaction problems [1] and overall user satisfaction.…”