Colocation for SLAM-Tracked VR Headsets with Hand Tracking

Reimer, Dennis; Podkosova, Iana; Scherzer, Daniel; Kaufmann, Hannes

doi:10.3390/computers10050058

Cited by 8 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While their method markedly improved the spatial consistency between collocated users, it was only applied to the homogeneous VR users who employed the HTC Vive system. Unlike the previous approaches based on either the fixed-point calibration or the markerbased calibration, Reimer et al [14] exploited hand tracking data as spatial anchors of colocated users wearing SLAMenabled VR headsets such as Meta Quest. For effective calibration between the colocated users, their method used an additional device capable of hand tracking.…”

Section: B: Alignment By Trackingmentioning

confidence: 99%

Integrating Heterogeneous VR Systems Into Physical Space for Collaborative Extended Reality

An,

Lee,

Park

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Virtual reality (VR) applications are typically developed within their own immersive digital worlds; therefore, virtual spaces are usually treated as discrete from the physical space where augmented and mixed reality users exist, which makes it difficult to combine these heterogeneous realities into an integrated extended reality (XR) environment. Along these lines, we propose a method that enables a user to geometrically register the virtual space within a VR application to a real space using a commodity camera in a workspace as an anchor point. We first investigate the mathematical aspect of the computational model for connecting the virtual space to the physical world. Then, we present a computational procedure that implements our proposed method with numerical accuracy and stability. As an application, we demonstrate that users of two VR systems from different vendors may collaborate within a shared real workspace while interacting with each other physically. The presented method provides a key mechanism for enabling XR users to leverage these immersive technologies by effectively using different realities within an integrated environment.INDEX TERMS Human-centered computing, Human computer interaction (HCI), Interaction paradigms, Virtual reality.

show abstract

Section: B: Alignment By Trackingmentioning

confidence: 99%

Integrating Heterogeneous VR Systems Into Physical Space for Collaborative Extended Reality

An,

Lee,

Park

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Problems ( 14) and (16) were solved in polynomial time by an adaptive voting algorithm [30]. The TLS rotation estimation (15) was relaxed to a tight semidefinite relaxation problem and quickly solved using the graduated non-convexity [31]. In addition, the estimation was used in a maximal clique inlier selection (MCIS) [32] after the scale estimation.…”

Section: Teaser++mentioning

confidence: 99%

“…These systems send key points and descriptors to the cloud for content localization. Certain solutions use visual landmarks for spatial anchors (data matrices, hand tracking) [15]. Such solutions have limitations because they require constant landmark visibility and calibration.…”

Section: Introductionmentioning

confidence: 99%

Comparison of Point Cloud Registration Algorithms for Mixed-Reality Cross-Device Global Localization

2023

View full text Add to dashboard Cite

State-of-the-art approaches for localization and mapping are based on local features in images. Along with these features, modern augmented and mixed-reality devices enable building a mesh of the surrounding space. Using this mesh map, we can solve the problem of cross-device localization. This approach is independent of the type of feature descriptors and SLAM used onboard the AR/MR device. The mesh could be reduced to the point cloud that only takes vertices. We analyzed and compared different point cloud registration methods applicable to the problem. In addition, we proposed a new pipeline Feature Inliers Graph Registration Approach (FIGRA) for the co-localization of AR/MR devices using point clouds. The comparative analysis of Go-ICP, Bayesian-ICP, FGR, Teaser++, and FIGRA shows that feature-based methods are more robust and faster than ICP-based methods. Through an in-depth comparison of the feature-based methods with the usual fast point feature histogram and the new weighted height image descriptor, we found that FIGRA has a better performance due to its effective graph-theoretic base. The proposed pipeline allows one to match point clouds in complex real scenarios with low overlap and sparse point density.

show abstract

“…Other examples of devices with inside-out tracking are the Microsoft Hololens and the Magic Leap. Some further works of interest demonstrate co-location for SLAMtracked VR headsets with hand tracking (Reimer et al, 2021) and motion capturing with a combination of inertial sensors and a camera (Yi et al, 2023). They obtain 3D positional data by implementing real-time room mapping and positional tracking that is based on visual-inertial simultaneous localization and mapping (viSLAM) techniques.…”

Section: Related Workmentioning

confidence: 99%

Tracking and Co-Location of Global Point Clouds for Large-Area Indoor Environments

Michiels,

Jorissen,

Put

et al. 2023

Preprint

View full text Add to dashboard Cite

Extended Reality (XR) experiences are on the verge of becoming widely adopted in diverse application domains. An essential part of the technology is accurate tracking and localization of the headset to create an immersive experience. A subset of the application domains requires perfect co-location between the real and the virtual world, where virtual objects are aligned with real-world counterparts. Current headsets support co-location for small areas, but suffer from drift when scaling up to larger ones such as buildings or factories. This paper proposes tools and solutions for this challenge by splitting up the simultaneous localization and mapping (SLAM) into separate mapping and localization stages. In the pre-processing stage, a feature map is built for the entire tracking area. A global optimizer is applied to correct the deformations caused by drift, guided by a sparse set of ground truth markers in the point cloud of a laser scan. Optionally, further refinement is applied by matching features between the ground truth keyframe images and their rendered-out SLAM estimates of the point cloud. In the second, real-time stage, the rectified feature map is used to perform localization and sensor fusion between the global tracking and the headset. The results show that the approach achieves robust co-location between the virtual and the real 3D environment for large and complex tracking environments.

show abstract

Colocation for SLAM-Tracked VR Headsets with Hand Tracking

Cited by 8 publications

References 21 publications

Integrating Heterogeneous VR Systems Into Physical Space for Collaborative Extended Reality

Integrating Heterogeneous VR Systems Into Physical Space for Collaborative Extended Reality

Comparison of Point Cloud Registration Algorithms for Mixed-Reality Cross-Device Global Localization

Tracking and Co-Location of Global Point Clouds for Large-Area Indoor Environments

Contact Info

Product

Resources

About