A usability study of multimodal input in an augmented reality environment

The personal cockpit

“…With a few exceptions [80,118,162], natural interaction techniques have not been widely studied in AR environments. Our work aims to supplement these existing techniques with input from a ring device to provide a sense of naturalism while countering the drawbacks of direct interaction with virtual objects, such as low precision [124,199] and fatigue [91,94].…”

“…Aside from combining direct and indirect, or relative and absolute techniques, researchers have also explored 'multimodal' techniques that combine multiple methods of natural interaction [118,162]. Piumsomboon et al [162], for example, combine hand gestures for manipulating virtual objects seen through a HWD, with voice commands for mode selection.…”

“…Multi-modal interfaces, for instance, are highly relevant to our discussion on interaction in chapter 6. This area has recently been explored for supporting natural interaction methods [118,162] and may provide alternative means for effective dual-tier interaction in SAIs. Further expansions of SAIs using 3D interfaces is inevitable.…”

Adaptive multimodal interaction in mobile augmented reality: A conceptual framework

“…However, using multimodal interaction might face some problems such as user needs to spend time to learn how to use the combination of unimodal interface. MMI will also increase mental workload where user has to pay attention in choosing an appropriate interface type to complete the activity for a given task [8]. Particularly in mobile settings, unimodal interaction modes can suffer from limitations (e.g.…”

“…al [8] proposed a framework of AR Multimodal Interface system. The components include input components for capturing video, gesture and speech input recognizer, a fusion, AR scene generation, and manager modules for producing the AR output and give the feedback to the user.…”