3D scanning deformable objects with a single RGBD sensor

Dou, Mingsong; Taylor, Jonathan; Fuchs, Henry; Fitzgibbon, Andrew; Izadi, Shahram

doi:10.1109/cvpr.2015.7298647

Cited by 137 publications

(88 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach was originally designed for rigid scenes, and one of the more well-known examples is KinectFusion [15]. This approach was later extended to handle non-rigid objects by describing the deformation of objects with transformations of signed distance field [9,13,14]. These methods can generate surprisingly high quality 3D shapes, but may lack tracking stability with regards to, e.g., occlusions.…”

Section: Human Shape Reconstructionmentioning

confidence: 99%

ReMagicMirror: Action Learning Using Human Reenactment with the Mirror Metaphor

Dayrit

Kimura

Nakashima

et al. 2016

MultiMedia Modeling

View full text Add to dashboard Cite

Abstract. We propose ReMagicMirror, a system to help people learn actions (e.g., martial arts, dances). We first capture the motions of a teacher performing the action to learn, using two RGB-D cameras. Next, we fit a parametric human body model to the depth data and texture it using the color data, reconstructing the teacher's motion and appearance. The learner is then shown the ReMagicMirror system, which acts as a mirror. We overlay the teacher's reconstructed body on top of this mirror in an augmented reality fashion. The learner is able to intuitively manipulate the reconstruction's viewpoint by simply rotating her body, allowing for easy comparisons between the learner and the teacher. We perform a user study to evaluate our system's ease of use, effectiveness, quality, and appeal.

show abstract

Section: Human Shape Reconstructionmentioning

confidence: 99%

ReMagicMirror: Action Learning Using Human Reenactment with the Mirror Metaphor

Dayrit

Kimura

Nakashima

et al. 2016

MultiMedia Modeling

View full text Add to dashboard Cite

show abstract

“…The single‐camera system avoids the redundant calibration work and is more space and cost effective compared with the multicamera system. One type of single‐camera system reconstructs 3D human body from dynamic inputs (e.g., the user rotates in front of the sensor continuously during capture) . However, reconstruction from continuous deforming inputs requires excessive nonrigid registration and data preprocessing such as temporal denoising .…”

Section: Related Workmentioning

confidence: 99%

“…However, reconstruction from continuous deforming inputs requires excessive nonrigid registration and data preprocessing such as temporal denoising . The overprocessing potentially degrades reconstruction accuracy and tends to oversmooth the reconstructed surface . To avoid excessive data processing, Li et al, Wang et al, and Zhang et al, adopted semi‐nonrigid pose assumption, in which four to eight (could be more) static poses are captured at different angles to cover the full body, and partial scan meshes are then generated.…”

Section: Related Workmentioning

confidence: 99%

Accurate nonrigid 3D human body surface reconstruction using commodity depth sensors

Zhao

Younes

et al. 2018

Computer Animation & Virtual

View full text Add to dashboard Cite

In the last decade, 3D modeling techniques enjoyed a booming development in both hardware and software. High-end hardware generates high fidelity results, but the cost is prohibitive, whereas consumer-level devices generate plausible results for entertainment purposes but are not appropriate for medical uses. We present a cost-effective and easy-to-use 3D body reconstruction system using consumer-grade depth sensors, which provides reconstructed body shapes with a high degree of accuracy and reliability appropriate for medical applications. Our surface registration framework integrates the articulated motion assumption, global loop closure constraint, and a general as-rigid-as-possible deformation model. To enhance the reconstruction quality, we propose a novel approach to accurately infer skeletal joints from anatomical data using multimodality registration. We further propose a supervised predictive model to infer the skeletal joints for arbitrary subjects independent from anatomical data reference. A rigorous validation test has been conducted on real subjects to evaluate the reconstruction accuracy and repeatability. Our system has the potential to make accurate body surface scanning systems readily available for medical professionals and the general public. The system can be used to obtain additional health data derived from 3D body shapes, such as the percentage of body fat.

show abstract

“…These methods, however, primarily focus on static scenes. In case of dynamic scenes, even state-of-the-art RGBD based systems [Newcombe et al 2015;Dou et al 2015] struggle with highly dynamic motion and occlusion. Capturing 3D geometry of dynamic scenes remains very challenging.…”

Section: Related Workmentioning

confidence: 99%

Smash

2016

View full text Add to dashboard Cite

Figure 1: Starting from an input video of a collision sequence behind a curtain (top), SMASH reconstructs an accurate physically valid collision (bottom) using laws of rigid body physics for regularization. Note the reconstructed spin (i.e., angular velocity) of the objects. AbstractCollision sequences are commonly used in games and entertainment to add drama and excitement. Authoring even two body collisions in the real world can be difficult, as one has to get timing and the object trajectories to be correctly synchronized. After tedious trial-and-error iterations, when objects can actually be made to collide, then they are difficult to capture in 3D. In contrast, synthetically generating plausible collisions is difficult as it requires adjusting different collision parameters (e.g., object mass ratio, coefficient of restitution, etc.) and appropriate initial parameters. We present SMASH to directly read off appropriate collision parameters directly from raw input video recordings. Technically we enable this by utilizing laws of rigid body collision to regularize the problem of lifting 2D trajectories to a physically valid 3D reconstruction of the collision. The reconstructed sequences can then be modified and combined to easily author novel and plausible collisions. We evaluate our system on a range of synthetic scenes and demonstrate the effectiveness of our method by accurately reconstructing several complex real world collision events.

show abstract

3D scanning deformable objects with a single RGBD sensor

Cited by 137 publications

References 29 publications

ReMagicMirror: Action Learning Using Human Reenactment with the Mirror Metaphor

ReMagicMirror: Action Learning Using Human Reenactment with the Mirror Metaphor

Accurate nonrigid 3D human body surface reconstruction using commodity depth sensors

Smash

Contact Info

Product

Resources

About