A three-dimensional characterization space of software components for rapidly developing multimodal interfaces

Serrano, Marcos; Juras, David; Nigay, Laurence

doi:10.1145/1452392.1452421

Cited by 13 publications

(20 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For instance, when using the visual languages provided by HephaisTK (Dumas et al 2014) and OIDE (Serrano et al 2008), one has to describe multimodal interactions in terms of the CARE properties (Coutaz et al 1995). For the ICO (Navarre et al 2009) language, depicting interaction models requires a solid command of a formalism called Petri nets.…”

Section: The Warnings Seem To Be Overlookedmentioning

confidence: 99%

“…Some examples of the aforementioned UIMSs include Mudra (Hoste et al 2011), ICO (Navarre et al 2009), OIDE (Serrano et al 2008), HephaisTK (Dumas et al 2014), and NiMMiT (De Boeck et al 2007). They certainly accomplish their goal of facilitating the prototyping of multimodal systems, but they all share the same issue: Their domain-specific languages require the use of concepts that are unrelated with the event languages with which programmers used to implement interactive systems in real-world projects 1 .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A user study for comparing the programming efficiency of modifying executable multimodal interaction descriptions: a domain-specific language versus equivalent event-callback code

Cuenca

Bergh

Luyten

et al. 2015

Proceedings of the 6th Workshop on Evaluation and Usability of Programming Languages and Tools

View full text Add to dashboard Cite

The present paper describes an empirical user study intended to compare the programming efficiency of our proposed domain-specific language versus a mainstream event language when it comes to modify multimodal interactions. By concerted use of observations, interviews, and standardized questionnaires, we managed to measure the completion rates, completion time, code testing effort, and perceived difficulty of the programming tasks along with the perceived usability and perceived learnability of the tool supporting our proposed language. Based on this experience, we propose some guidelines for designing comparative user studies of programming languages. The paper also discusses the considerations we took into account when designing a multimodal interaction description language that intends to be well regarded by its users.

show abstract

Section: The Warnings Seem To Be Overlookedmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A user study for comparing the programming efficiency of modifying executable multimodal interaction descriptions: a domain-specific language versus equivalent event-callback code

Cuenca

Bergh

Luyten

et al. 2015

Proceedings of the 6th Workshop on Evaluation and Usability of Programming Languages and Tools

View full text Add to dashboard Cite

show abstract

“…Fig. 2 presents an example of a graphically specified multimodal interaction using OIDE [13]: the example involves the combined usage of speech and gesture for performing a zoom task on a map displayed on an augmented table.…”

Section: IImentioning

confidence: 99%

“…Figure 2. Combined usage of speech and gesture for performing a zoom task on a map displayed on an augmented table: designer assembly for specifying a zoom task and screenshot of the same assembly in the OIDE (adapted from [13]). …”

Section: IImentioning

confidence: 99%

Adaptable multimodal interfaces in pervasive environments

Avouac

Lalanda

Nigay

2012

2012 IEEE Consumer Communications and Networking Conference (CCNC)

Self Cite

View full text Add to dashboard Cite

Abstract-In the context of pervasive environments, multimodal interaction plays a pivotal role because multimodality provides flexibility and naturalness for interaction. The challenge of multimodal interfaces in pervasive environments is then to build reliable and autonomic processing systems able to analyze and understand multiple interaction modalities and reconfigure itself in real-time. Addressing this issue, we have developed an autonomic framework called DynaMo (Dynamic multiMOdality) for the development and runtime management of multimodal interaction in pervasive environments. DynaMo is composed by a specification language dedicated to the multimodality domain and a runtime machine that instantiates these specifications. In this paper, we present the overall architecture of our solution DynaMo that is based on partial interaction models, and how these models are completed at runtime to build multimodal interfaces adapted to the local execution environment.

show abstract

“…To do so, we define WoZ components as parts of the component-based approach for rapidly developing multimodal prototypes defined in [26]. These WoZ components are characterized according to the roles that a WoZ component can play in the data-flow of input multimodal interaction, from devices to tasks.…”

mentioning

confidence: 99%

A wizard of oz component-based approach for rapidly prototyping and testing input multimodal interfaces

Serrano

Nigay

2010

J Multimodal User Interfaces

Self Cite

View full text Add to dashboard Cite

In this paper we present a novel approach for prototyping, testing and evaluating multimodal interfaces, OpenWizard. OpenWizard allows the designer and the developer to rapidly evaluate a non-fully functional multimodal prototype by replacing one modality or a composition of modalities that are not yet available by wizard of oz techniques. OpenWizard is based on a conceptual componentbased approach for rapidly developing multimodal interfaces, an approach first implemented in the ICARE software tool and more recently in the OpenInterface tool. We present a set of wizard of oz components that are implemented in OpenInterface. While some wizard of oz (WoZ) components are generic to be reused for different multimodal applications, our approach allows the integration of tailored WoZ components. We illustrate OpenWizard using a multimodal map navigator.

show abstract

A three-dimensional characterization space of software components for rapidly developing multimodal interfaces

Cited by 13 publications

References 28 publications

A user study for comparing the programming efficiency of modifying executable multimodal interaction descriptions: a domain-specific language versus equivalent event-callback code

A user study for comparing the programming efficiency of modifying executable multimodal interaction descriptions: a domain-specific language versus equivalent event-callback code

Adaptable multimodal interfaces in pervasive environments

A wizard of oz component-based approach for rapidly prototyping and testing input multimodal interfaces

Contact Info

Product

Resources

About