Multimodal user interfaces in the Open Agent Architecture

Moran, Douglas B.; Cheyer, Adam; Julia, Luc

doi:10.1145/238218.238290

Cited by 47 publications

(26 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The Open Agent Architecture (OAA) architecture was designed to ease the implementation of agent-based applications, enabling intelligent, cooperative, distributed, and multimodal agentbased user interfaces (Moran et al, 1997). The agents can be developed in several high-level languages (e.g., C or Java) and platforms (e.g., Windows and Solaris).…”

Section: Architectures For the Design Of Multimodal Interfacesmentioning

confidence: 99%

“…The cooperation and communication between the agents is carried out by means of an agent called Facilitator. Several authors have used this architecture to implement multimodal interfaces for a variety of application domains, including map-based tourist information (Moran et al, 1997), interaction with robots (Bos et al, 2003), and control of user movements in a 2D game (Corradini & Samuelsson, 2008).…”

Section: Architectures For the Design Of Multimodal Interfacesmentioning

confidence: 99%

See 1 more Smart Citation

On the Development of Adaptive and User-Centred Interactive Multimodal Interfaces

Griol

Callejas

López-Cózar

et al. 2012

Speech, Image, and Language Processing for Human Computer Interaction

View full text Add to dashboard Cite

Multimodal systems have attained increased attention in recent years, which has made possible important improvements in the technologies for recognition, processing, and generation of multimodal information. However, there are still many issues related to multimodality which are not clear, for example, the principles that make it possible to resemble human-human multimodal communication. This chapter focuses on some of the most important challenges that researchers have recently envisioned for future multimodal interfaces. It also describes current efforts to develop intelligent, adaptive, proactive, portable and affective multimodal interfaces.

show abstract

Section: Architectures For the Design Of Multimodal Interfacesmentioning

confidence: 99%

Section: Architectures For the Design Of Multimodal Interfacesmentioning

confidence: 99%

On the Development of Adaptive and User-Centred Interactive Multimodal Interfaces

Griol

Callejas

López-Cózar

et al. 2012

Speech, Image, and Language Processing for Human Computer Interaction

View full text Add to dashboard Cite

show abstract

“…OAA has been used to implement more than 30 applications in various domains, many of them multimodal in nature [5]. OAA has also been used by organisations outside SRI.…”

Section: Integration Frameworkmentioning

confidence: 99%

“…Alternatives for doing so include grammarbased approaches, where an engineer explicitly defines the 'tree' of possible utterances, and statistical models computed across a specialised corpus of collected speech in the target domain. Nuance Communications 5 , one of the leaders in speechenabled telephony applications, uses grammars to empower telephone-based applications for stock quote retrieval or accessing email and calendar information. IBM's ViaVoice developer tools include compilers and APIs for using speech grammars.…”

Section: Introductionmentioning

confidence: 99%

Spoken language and multimodal applications for electronic realities

Cheyer

Julia

1999

Virtual Reality

View full text Add to dashboard Cite

Abstract:We use the term 'electronic reality' (ER) to encompass a broad class of concepts that mix real-world metaphors and computer interfaces. In our definition, 'electronic reality' includes notions such as virtual reality, augmented reality, computer interactions with physical devices, interfaces that enhance 2D media such as paper or maps, and social interfaces where computer avatars engage humans in various forms of dialogue. One reason for bringing real-world metaphors to computer interfaces is that people already know how to navigate and interact with the world around them. Every day, people interact with each other, with pets, and sometimes with physical objects by using a combination of expressive modalities, such as spoken words, tone of voice, pointing and gesturing, facial expressions, and body language. In contrast, when people typically interact with computers or appliances, interactions are unimodal, with a single method of communication such as the click of a mouse or a set of keystrokes serving to express intent. In this article, we describe our efforts to apply multimodal and spoken language interfaces to a number of ER applications, with the goal of creating an even more 'realistic' or natural experience for the end user.

show abstract

“…Going one step further than providing a particular modality or generic reusable mechanisms (i.e. fusion and mutual disambiguation mechanisms), Quickset [Johnston et al 1997] defines an overall implementation architecture as well as the Open Agent Architecture (OAA) [Moran et al 1997]. Quickset mainly focuses on input multimodality based on speech and gesture and has been applied to the development of map-based systems.…”

Section: Related Work: Tools For Multimodalitymentioning

confidence: 99%

Output Multimodal Interaction: The Case of Augmented Surgery

Mansoux

Nigay²,

Troccaz

People and Computers XX — Engage

View full text Add to dashboard Cite

Output multimodal interaction involves choice and combination of relevant interaction modalities to present information to the user. In this paper, we present a framework based on reusable software components for rapidly developing output multimodal interfaces by choosing and combining interaction modalities. Such an approach enables us to quickly explore several design alternatives as part of an iterative design process. Our approach is illustrated by examples from a computer-assisted surgery system that runs in a specific environment (i.e. an operating room) and so needs adapted multimodal interaction. Our approach supports the exploration of several output multimodal interaction design alternatives with the surgeons.

show abstract

Multimodal user interfaces in the Open Agent Architecture

Cited by 47 publications

References 10 publications

On the Development of Adaptive and User-Centred Interactive Multimodal Interfaces

On the Development of Adaptive and User-Centred Interactive Multimodal Interfaces

Spoken language and multimodal applications for electronic realities

Output Multimodal Interaction: The Case of Augmented Surgery

Contact Info

Product

Resources

About