Comparing paper and tangible, multimodal tools

Abstract

In command posts, officers maintain situational awareness using paper maps. Post-it notes, and hand-written annotations. They do so because paper is robust to failure, it is portable, it offers a flexible means of capturing information, it has ultra-high resolution, and it readily supports face-to-face collaboration. We report herein on an evaluation comparing maps and Post-its with a tangible multimodal system called Rasa. Rasa augments these paper tools with sensors, enabling it to recognize the multimodal language (both written and spoken) that naturally occurs on them. In this study, we found that not only do users prefer Rasa to paper alone, they find it as easy or easier to use than paper tools. Moreover, Rasa introduces no discernible overhead in its operation other than error repair, yet grants the benefits inherent in digital systems. Finally, subjects confirmed that by combining physical and computational tools. Rasa is resistant to computational failure.

Download Complete Issue: CP54 (27MB)

“…In some contexts, such as crisis response and military planning, large paper maps retain a distinct advantage in their combination of high resolution and portability-even in comparison to physically augmented virtual spaces such as the EDC described above. As noted in the previous section, McGee and colleagues (McGee et al, 2002;McGee et al, 2000;McGee et al, 2001) have studied military planners working with such maps (in both wall mounted and table top situations). Based on this research, they proposed an approach to augmenting paper maps through digital Post-it © notes (physical notes for which the position and content of the note could be sensed by the system).…”

Speech-gesture driven multimodal interfaces for crisis management

“…The crisis management context provides a particular challenge for development of integrated speech-gesture interfaces, since an important component of crisis management (response) is typically carried out under conditions of considerable stress. Although there is a growing body of research on speech-gesture interfaces to geospatial information (usually presented via maps) [5]- [9], little attention has been directed thus far to specific challenges of developing these interfaces to cope with interaction in stressful situations. We believe that an integrated, multidisciplinary approach is necessary to understanding the ill-structured, highly dynamic, collaborative work domain of crisis management for the design of multimodal systems.…”