An Exploration of Freehand Crossing Selection in Head-Mounted Augmented Reality

Abstract

Crossing, or goal crossing, has proven useful in various selection scenarios, including pen, mouse, touch, and virtual reality (VR). However, crossing has not been exploited for freehand selection using augmented reality head-mounted displays (AR HMDs). Using the HoloLens, we explore freehand crossing for selection and compare it to the state-of-the-art “gaze and commit” (head gaze) method. We report on three studies investigating freehand crossing in multiple use cases. The first study shows that crossing outperforms head gaze in selection scenarios of varying target arrangements. The second explores crossing, head gaze, and hand pointing in radial menu and dynamic interface scenarios. The third explores crossing as a function carrier for a variety of basic interaction functions in a drawing application. This work builds on existing knowledge on the goal-crossing paradigm by demonstrating its potential as a useful interaction method in 3D AR HMD interfaces.

Quick Porting of Virtual Simulation Experiment Interactions from the Web 3D to VR

“…Tung and Hertzmann (2018) noted that users actually perform many 2D tasks in a 3D environment, such as buttons, sliders, and other widgets. Uzor and Kristensson (2021) found that 2D image planes have been shown to facilitate relatively simple 2D choices in a 3D environment. Therefore, it is feasible to use 2DUI in a virtual environment.…”

“…Research on designing user interface (UI) interaction is a major focus in VR development. Many studies have been conducted to compare different interaction modes of VR (Argelaguet and Andujar, 2013;Bergström et al, 2021;Uzor and Kristensson, 2021), however, designing such modes specifically for the VR environment is time-consuming. Currently, many 3D scenes are available on the web.…”

“…Ray and touch interactions are two of the most developed and commonly used modes of interaction in commercial systems, such as the HTC Vive and Oculus Rift (Uzor and Kristensson, 2021;Wagner et al, 2021). Ray interaction is accurate over short distances although targets can also be selected over long distances.…”

Freehand Gestural Selection with Haptic Feedback in Wearable Optical See-Through Augmented Reality

“…Other input modalities, such as head motion, hand gesture, and eye gazing, could also be used to point and select small remote targets [10]. Besides virtual hands and pointing selection techniques, other selection methods, such as goal-crossing selection enabled by freehand tracking of the Microsoft Hololens, have also been evaluated in OST AR environments [9].…”

“…For example, selecting a target is one of the most-frequent user interaction tasks. Target selection has been extensively studied in a variety of interaction contexts, including desktop computers [3,4], multi-touch mobile devices [5], 3D displays with hand-held devices [6], or freehand gestures [7][8][9][10]. In particular, freehand gestural interaction, which could provide a direct and natural user experience, is already expanding outside lab settings and is commonly supported by commercial wearable OST AR products.…”