Real-Time 3D Reconstruction for Mixed Reality Telepresence Using Multiple Depth Sensors

Ishigaki, Shafina Abd Karim; Ismail, Ajune Wanis

doi:10.1007/978-3-031-25088-0_5

Cited by 3 publications

(1 citation statement)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach significantly reduces bandwidth requirements compared to previous voxelbased methods. In [13], a method for producing a single full 3D reconstruction surface of a moving user in real-time is introduced by using multiple depth sensors and a marching square approach to produce a single full 3D reconstruction surface. This method has been specifically designed for Mixed Reality (MR) telepresence.…”

Section: D Live Reconstruction For Mixed Realitymentioning

confidence: 99%

Saliency-Guided Point Cloud Compression for 3D Live Reconstruction

Ruiu,

Mascia,

Grosso

2024

MTI

View full text Add to dashboard Cite

3D modeling and reconstruction are critical to creating immersive XR experiences, providing realistic virtual environments, objects, and interactions that increase user engagement and enable new forms of content manipulation. Today, 3D data can be easily captured using off-the-shelf, specialized headsets; very often, these tools provide real-time, albeit low-resolution, integration of continuously captured depth maps. This approach is generally suitable for basic AR and MR applications, where users can easily direct their attention to points of interest and benefit from a fully user-centric perspective. However, it proves to be less effective in more complex scenarios such as multi-user telepresence or telerobotics, where real-time transmission of local surroundings to remote users is essential. Two primary questions emerge: (i) what strategies are available for achieving real-time 3D reconstruction in such systems? and (ii) how can the effectiveness of real-time 3D reconstruction methods be assessed? This paper explores various approaches to the challenge of live 3D reconstruction from typical point cloud data. It first introduces some common data flow patterns that characterize virtual reality applications and shows that achieving high-speed data transmission and efficient data compression is critical to maintaining visual continuity and ensuring a satisfactory user experience. The paper thus introduces the concept of saliency-driven compression/reconstruction and compares it with alternative state-of-the-art approaches.

show abstract

Section: D Live Reconstruction For Mixed Realitymentioning

confidence: 99%