Head-Controlled Menu in Mixed Reality with a HMD

Abstract

We present a design-space and three new techniques for headbased interaction with menus in Mixed Reality (MR) with a Head-Mounted Display (HMD). Usual input modalities such as hand gestures and voice commands are not suitable in noisy MR contexts where the users have both hands occupied as in augmented surgery and machine maintenance. To address the two issues of noisy MR contexts and hand-free interaction, we systematically explore the design space of headcontrolled menu interaction by considering two design factors: 1) headcontrolled menu versus head-controlled cursor 2) virtual targets versus mixed targets anchored on physical objects. Based on the design space, we present three novel menu techniques that we compared with a baseline head-controlled cursor technique. Experimental results suggest that head-controlled menu and head-controlled cursor techniques offer similar performance. In addition, the study found that mixed targets do not impact ultimate user performance when users are trained enough, but improve the learning phase. When using virtual targets, users still progressed after the training phase by reducing their mean selection time by 0.84s. When using mixed targets, the improvement was limited to 0.3s.

Bring2Me

“…We considered that if a hand is available at the moment of the interaction, it should be used to perform eyes-free pointing and preserve the visual context. Therefore, we chose a standard head-controlled cursor technique [5,18,23] which is eyes-engaged and does not require gestures with the hand. A cross-shaped cursor is displayed at a fixed position at the center of the HMD display, and has to be aligned with the target by turning the head.…”

“…Using a hand-held tool as the origin of the ray is fairly common [6,13]. Since MR with a HMD implies that the augmented FOV is directly linked to the position of the head, several works also explored head-based raycasting techniques [5,18]. In these studies, the position of the cursor is directly linked to the position of the head.…”

“…Inspired by previous work on pointing task in MR [5], a textual guidance indicated the current menu and item to be selected at the beginning of a trial (e.g. "Middle Left, K" in Figure 3b).…”

“…Such a scenario can be found in many contexts, including augmented surgery. As explained by Bailly et al, surgeons are used to interact with menus during an operation [5]. By wearing a HMD, surgeons no longer rely on a single physical screen in the operating room: virtual menus can be located across the whole operating theatre.…”

“…This is a strong issue when users need to focus their visual attention on their main task. Surgeons looking away from the patient have to interrupt their surgical task, which adds cognitive load to already busy-minded users [5]. Alternative solutions to interact with virtual widgets that preserve users' visual context are required.…”