Dynamic adaptive mesh streaming for real-time 3D teleimmersion

Abstract

is an open access repository that collects the work of Arts et Métiers ParisTech researchers and makes it freely available over the web where possible. AbstractRecent advances in full body 3D reconstruction methods have lead to the realisation of high quality, real-time, photo realistic capture of users in a range of tele-immersion (TI) contexts including gaming and mixed reality environments. The full body reconstruction (FBR) process is computationally expensive requiring comparatively high CPU, GPU and network resources in order to maintain a shared, virtual reality in which high quality 3D reproductions of users can be rendered in real-time. A significant optimisation of the delivery of FBR content has been achieved through the realtime compression and de-compression of 3D geometry and textures. Here we present a new, adaptive compression methodology that allows a TI system called 3D-LIVE to modify the quality and speed of a FBR TI pipeline based on the data carrying capability of the network. Our rule-based adaptation strategy uses network performance sampling processes and a configurable rule engine to dynamically alter the compression of FBR reconstruction on-the-fly. We demonstrate the efficacy of the approach with an experimental evaluation of system and conclude with a discussion of future directions for adaptive FBR compression.

An Integrated Platform for Live 3D Human Reconstruction and Motion Capturing

“…This paper describes the main elements of an integrated platform, including capturing and fast 3D reconstruction of human 3D shape/appearance and skeleton-based motion tracking, which targets TI and future 3D applications. The elements of the continuously being developed platform have already allowed the realization of a number of relevant applications, as in http://vcl.iti.gr/3dTI/ : 1) ski competition among users spread around Europe [3]; 2) 3D hang-out communications [4]; 3) multiplayer networked 3D games ("SpaceWars" and "Castle in the Forest"), where users participate via their on-the-fly reconstructed 3D replicants; and 4) athletes' training via professionals' performance capturing and reconstruction for quick-post 4D media. In addition, this paper describes a novel framework for the objective evaluation of the 3D reconstruction process, where the 3D ground-truth model is not available, as in real-time reconstruction applications.…”