Detection-Guided 3D Hand Tracking for Mobile AR Applications

Che, Yunlong; Qi, Yue

doi:10.1109/ismar52148.2021.00055

Cited by 4 publications

(1 citation statement)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They used OpenPose for 2D hand recognition and non-linear least-squares minimization to fit a 3D model of the hand to the estimated 2D joint positions, recovering the 3D hand pose. Additional recognition methods were developed by Che and Qi (2021) and Sun et al (2021) recognizing the hand region and deriving 2D and 2.5D points of the hand using networks. Wang et al (2020) presented a method to track 3D realtime interactions with a monocular RGB camera.…”

Section: Hand Tracking With Rgb and Rgb-d Camerasmentioning

confidence: 99%

Evaluation and improvement of HMD-based and RGB-based hand tracking solutions in VR

Reimer

Podkosova

Scherzer

et al. 2023

Front. Virtual Real.

View full text Add to dashboard Cite

Hand tracking has become a state-of-the-art technology in the modern generation of consumer VR devices. However, off-the-shelf solutions do not support hand detection for more than two hands at the same time at distances beyond arm’s length. The possibility to track multiple hands at larger distances would be beneficial for colocated multi-user VR scenarios, allowing user-worn devices to track the hands of other users and therefore reducing motion artifacts caused by hand tracking loss. With the global focus of enabling natural hand interactions in colocated multi-user VR, we propose an RGB image input-based hand tracking method, built upon the MediaPipe framework, that can track multiple hands at once at distances of up to 3 m. We compared our method’s accuracy to that of Oculus Quest and Leap Motion, at different distances from the tracking device and in static and dynamic settings. The results of our evaluation show that our method provides only slightly less accurate results than Oculus Quest or Leap motion in the near range (with median errors below 1.75 cm at distances below 75 cm); at larger distances, its accuracy remains stable (with a median error of 4.7 cm at the distance of 2.75 m) while Leap Motion and Oculus Quest either loose tracking or produce very inaccurate results. Taking into account the broad choice of suitable hardware (any RGB camera) and the ease of setup, our method can be directly applied to colocated multi-user VR scenarios.

show abstract

Section: Hand Tracking With Rgb and Rgb-d Camerasmentioning

confidence: 99%