VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction

Innmann, Matthias; Zollhöfer, Michael; Nießner, Matthias; Theobalt, Christian; Stamminger, Marc

doi:10.1007/978-3-319-46484-8_22

Cited by 276 publications

(337 citation statements)

References 44 publications

Supporting

Mentioning

336

Contrasting

Unclassified

Order By: Relevance

“…Even real-time tracking of general deforming objects [Zollhöfer et al 2014] and template-free reconstruction [Dou et al 2016;Innmann et al 2016;Newcombe et al 2015; has been demonstrated. RGB-D information overcomes forwardbackwards ambiguities in monocular pose estimation.…”

Section: Multi-viewmentioning

confidence: 99%

VNect

et al. 2017

Self Cite

View full text Add to dashboard Cite

Fig. 1. We recover the full global 3D skeleton pose in real-time from a single RGB camera, even wireless capture is possible by streaming from a smartphone (left). It enables applications such as controlling a game character, embodied VR, sport motion analysis and reconstruction of community video (right). Community videos (CC BY) courtesy of Real Madrid C.F. [2016] and RUSFENCING-TV [2017].We present the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera. Our method combines a new convolutional neural network (CNN) based pose regressor with kinematic skeleton fitting. Our novel fullyconvolutional pose formulation regresses 2D and 3D joint positions jointly in real time and does not require tightly cropped input frames. A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton. This makes our approach the first monocular RGB method usable in real-time applications such as 3D character control-thus far, the only monocular methods for such applications employed specialized RGB-D cameras. Our method's accuracy is quantitatively on par with the best offline 3D monocular RGB pose estimation methods. Our results are qualitatively comparable to, and sometimes better than, results from monocular RGB-D approaches, such as the Kinect. However, we show that our approach is more broadly applicable than RGB-D solutions, i.e., it works for outdoor scenes, community videos, and low quality commodity RGB cameras.

show abstract

Section: Multi-viewmentioning

confidence: 99%

VNect

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Some approaches [21,22,29,30,49] have recently been proposed for reconstruction of non-rigid objects and scenes using a single RGBD sensor. Izadi et al [22] and Newcombe et al [30] introduced KinectFusion as a real-time 3D reconstruction approach using a moving Kinect.…”

Section: Related Workmentioning

confidence: 99%

3D Data Acquisition and Registration Using Two Opposing Kinects

Soleimani

Mirmehdi

Damen

et al. 2016

2016 Fourth International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

General rightsThis document is made available in accordance with publisher policies. Please cite only the published version using the reference above. AbstractWe present an automatic, open source data acquisition and calibration approach using two opposing RGBD sensors (Kinect V2) and demonstrate its efficacy for dynamic object reconstruction in the context of monitoring for remote lung function assessment. First, the relative pose of the two RGBD sensors is estimated through a calibration stage and rigid transformation parameters are computed. These are then used to align and register point clouds obtained from the sensors at frame level. We validated the proposed system by performing experiments on known-size box objects with the results demonstrating accurate measurements. We also report on dynamic object reconstruction by way of human subjects undergoing respiratory functional assessment.

show abstract

“…This approach was originally designed for rigid scenes, and one of the more well-known examples is KinectFusion [15]. This approach was later extended to handle non-rigid objects by describing the deformation of objects with transformations of signed distance field [9,13,14]. These methods can generate surprisingly high quality 3D shapes, but may lack tracking stability with regards to, e.g., occlusions.…”

Section: Human Shape Reconstructionmentioning

confidence: 99%

ReMagicMirror: Action Learning Using Human Reenactment with the Mirror Metaphor

Dayrit

Kimura

Nakashima

et al. 2016

MultiMedia Modeling

View full text Add to dashboard Cite

Abstract. We propose ReMagicMirror, a system to help people learn actions (e.g., martial arts, dances). We first capture the motions of a teacher performing the action to learn, using two RGB-D cameras. Next, we fit a parametric human body model to the depth data and texture it using the color data, reconstructing the teacher's motion and appearance. The learner is then shown the ReMagicMirror system, which acts as a mirror. We overlay the teacher's reconstructed body on top of this mirror in an augmented reality fashion. The learner is able to intuitively manipulate the reconstruction's viewpoint by simply rotating her body, allowing for easy comparisons between the learner and the teacher. We perform a user study to evaluate our system's ease of use, effectiveness, quality, and appeal.

show abstract

VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction

Cited by 276 publications

References 44 publications

VNect

VNect

3D Data Acquisition and Registration Using Two Opposing Kinects

ReMagicMirror: Action Learning Using Human Reenactment with the Mirror Metaphor

Contact Info

Product

Resources

About