Pocket6

Abstract

We propose, implement and evaluate the use of a smartphone application for real-time six-degrees-of-freedom user input. We show that our app-based approach achieves high accuracy and goes headto-head with expensive externally tracked controllers. The strength of our application is that it is simple to implement and is highly accessible -requiring only an off-the-shelf smartphone, without any external trackers, markers, or wearables. Due to its inside-out tracking and its automatic remapping algorithm, users can comfortably perform subtle 3D inputs everywhere (world-scale), without any spatial or postural limitations. For example, they can interact while standing, sitting or while having their hands down by their sides. Finally, we also show its use in a wide range of applications for 2D and 3D object manipulation, thereby demonstrating its suitability for diverse real-world scenarios.

Exploring Users' Pointing Performance on Virtual and Physical Large Curved Displays

“…Prior work on target pointing mostly considered linear (e.g., in meters or pixels) target width and amplitude [5,20,24,42,45,51,63,66]. Later, angular measurements (i.e., in degrees or radians) of target width and amplitude were explored for pointing tasks [35][36][37].…”

“…Numerous studies have used Fitts law to explore users' pointing performance on large physical displays [5,24,42,45,51,63,66]. These studies revealed that users' performance is significantly affected by target width and amplitude (i.e., the distance between two targets in meters or pixels).…”

Understanding and Creating Spatial Interactions with Distant Displays Enabled by Unmodified Off-The-Shelf Smartphones

“…Using only the smartphone to enable the interaction with a distant display allows users to use their fingers for touch inputs and movement of their device-holding hand for hand motion input. These device-tracking inputs are enabled by the device's touchscreen [5] and pose tracking (i.e., device tilting [1,24] as orientation, motion as translation [3,25], or pose as translation + orientation [2]), while touch-input and tilting the device represent the majority of the past as well as recent papers, we can see a trend towards utilizing SLAM [26] that is natively implemented on modern smartphones, e.g., ARCore [27], ARKit [28]. These libraries show how SLAM can be used for precisely tracking the absolute worldspace pose of the smartphone (i.e., the motion of the user's device-holding hand) in distant display scenarios [2].…”

“…These device-tracking inputs are enabled by the device's touchscreen [5] and pose tracking (i.e., device tilting [1,24] as orientation, motion as translation [3,25], or pose as translation + orientation [2]), while touch-input and tilting the device represent the majority of the past as well as recent papers, we can see a trend towards utilizing SLAM [26] that is natively implemented on modern smartphones, e.g., ARCore [27], ARKit [28]. These libraries show how SLAM can be used for precisely tracking the absolute worldspace pose of the smartphone (i.e., the motion of the user's device-holding hand) in distant display scenarios [2]. However, approaches that would extend currently known device-tracking inputs for body-tracking of the user's body, head pose as well as gaze still remain undiscovered (see Supplementary Materials).…”

“…For decades, researchers investigated the way how we are interacting with distant displays by using mobile devices [1][2][3]. Initially, users controlled distant displays via buttons on a remote controller or later a touchscreen device.…”