Reliable Evaluation of Multimodal Dialogue Systems

Metze, Florian; Wechsung, Ina; Schaffer, Stefan; Seebode, Julia; Möller, Sebastian

doi:10.1007/978-3-642-02577-8_9

Cited by 14 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ren et al [8] evaluated the suitability of the multimodal combination of Mouse, Keyboard, Speech and Pen for map and CAD applications. Metze et al [9] evaluated the multimodal combinations of touch and speech modalities on a wall mounted GUI based room management system. Bernhaupt et al [10] evaluated the multimodal combination of two mouse and speech on an industry grade safety critical system.…”

Section: Related Workmentioning

confidence: 99%

Exploration of multimodal input interaction based on goals

Ghosh¹,

Joshi

2013

Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction

View full text Add to dashboard Cite

Today applications demand newer and newer ways through which humans can interact with the system for effective and natural interaction. Each interaction modality have unique features and supports different interaction goals, which makes them logically appropriate for specific types of interaction necessary for different applications. With several multimodal input technologies now being available for users, a challenge is to design optimal combinations of multiple modalities for specific tasks. Also, it is important to understand that, how these modalities may coherently be used together in a well-coordinated manner. This paper presents our initial research wherein we did exploration with the objective of designing a scheme for identification of optimal multimodal input interaction for various applications and use cases. We identified the key interaction goals of few sample applications or usage scenarios, and systematically compared those with the capabilities of the modalities, in order to identify optimal multimodal combination for those applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Exploration of multimodal input interaction based on goals

Ghosh¹,

Joshi

2013

Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction

View full text Add to dashboard Cite

show abstract

“…Broad categories work related to the usability evaluation of multimodal interaction includes, multimodal evaluations through user questionnaires [3], user performance logs [4] [5], both performance logs along with questionnaires [6], Wizard of Oz technique [7] [8], eye tracking [9], model based formal verification methods with the use of Petri-nets [10] or Finite State Machines [11], etc. Ren et al [5] reported empirical evaluation of Mouse, Keyboard, Speech and Pen for a prototype map and CAD applications.…”

Section: Related Workmentioning

confidence: 99%

“…Ren et al [5] reported empirical evaluation of Mouse, Keyboard, Speech and Pen for a prototype map and CAD applications. Metze et al [6] used the post experiment user questionnaires to compare touch and speech modalities independently and their multimodal combination, for a wall mounted GUI based room management system. Similarly, Kaster et al [12] evaluated the performance of the uni-modal combinations (only a single modality) and bi-modal combinations (two modalities) of mouse with speech, and touch with speech.…”

Section: Related Workmentioning

confidence: 99%

Empirical Evaluation of Multimodal Input Interactions

Ghosh

Joshi

Tripathi³

2013

Human Interface and the Management of Information. Information and Interaction Design

View full text Add to dashboard Cite

With variety of interaction technologies like speech, pen, touch, hand or body gestures, eye gaze, etc., being now available for users, it is a challenge to design optimal and effective multimodal combinations for specific tasks. For designing that, it is important to understand how these modalities can be combined and used in a coordinated manner. We performed an experimental evaluation of combinations of different multimodal inputs, such as keyboard, speech and touch with pen etc, in an attempt to investigate, which combinations are efficient for diverse needs of the users. In our study, multimodal combination of three modalities was found to be more effective in terms of performance, accuracy and user experience than that of two modalities. Further, we also inferred the roles that each of the modalities play in a multimodal combination to achieve the usability goals.

show abstract