A Hybrid 2D-3D Tangible Interface for Virtual Reality

Zhang, Li; He, Weiping; Bai, Huidong; He, Jun; Qiao, Yiyue; Billinghurst, Mark

doi:10.1145/3450618.3469166

Cited by 4 publications

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Early work in this area explored the idea of connecting a smartphone to a VR or Augmented Reality (AR) system and allowing the user to interact with 3D objects using a combination of HMD's and Hand-Held Devices (HHD's) [7,8]. A few different approaches using a smartphone or other smart devices for interaction in VR and AR have been proposed since, with and without some type of external tracking or hardware adaptations [1][2][3][4][5][6][9][10][11][12][13][14][15]. While some approaches focus on using built-in sensors in smart devices to control hand held ray casters or pointers [16][17][18], our approach is mainly image-based, like TrackCap and PAIR [4,5].…”

Section: Related Workmentioning

confidence: 99%

“…This would allow users to use a device they already own and are familiar with, which reduces the cost of the system and makes VR more accessible to the average consumer. Moreover, smartphones contain additional sensors and displays which can also be used to improve the interaction capabilities of regular VR systems, as suggested in by previous researchers [1][2][3][4][5][6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SmartVR Pointer: Using Smartphones and Gaze Orientation for Selection and Navigation in Virtual Reality

McDonald,

Zhang,

Nanzatov

et al. 2024

Preprint

View full text Add to dashboard Cite

Some of the barriers preventing Virtual Reality (VR) from being widely adopted are the cost and unfamiliarity of VR systems. Here we propose that in many cases the specialized controllers shipped with most VR head-mounted displays can be replaced by a regular smartphone, cutting the cost of the system, and allowing users to interact in VR using a device they are already familiar with. To achieve this, we developed SmartVR Pointer, an approach that uses smartphones as a replacement for the specialized controllers for two essential operations in VR: selection and navigation by teleporting. In SmartVR Pointer a camera mounted on the head-mounted display (HMD) is tilted downwards so that it points to where the user will naturally be holding their phone in front of them. SmartVR Pointer supports three selection modalities: tracker-based,gaze-based, and combined/hybrid. In the tracker-based SmartVR pointer selection we use image-based tracking to track a QR code displayed on the phone screen and them map the phone’s position to a pointer shown within the field of view of the camera in the Virtual Environment. In the gaze-based selection modality the user places the pointer using their gaze and taps on the phone for selection. The combined technique is a hybrid between head-gaze-based interaction in VR and smartphone-based Augmented Reality. It allows the user to control a VR pointer that behaves like a mouse pointer by moving their smartphone to select objects within the virtual environment, and to interact with the selected objects using the smartphone’s touch screen. The touchscreen is used for selection and dragging. SmartVR Pointer is simple and requires no calibration and no complex hardware assembly or disassembly. We demonstrate successful interactive applications of SmartVR Pointer in a VR environment with a demo where the user navigates in the virtual environment using teleportation points on the floor and then solves a Tetris-style key-and-lock challenge.

show abstract