Animation cartography—intrinsic reconstruction of shape and motion

Tevs, Art; Berner, Alexander; Wand, Michael; Ihrke, Ivo; Bokeloh, Martin; Kerber, Jens; Seidel, Hans‐Peter

doi:10.1145/2159516.2159517

Cited by 78 publications

(50 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Aligning these scans is a challenging task that often requires high quality scan data and small changes of the pose in each scan [3] [21]. While most of these approaches mainly deal with high-resolution scan data, we propose a robust nonrigid global alignment that can work well even with the Kinect's resolution and noise levels.…”

Section: Related Workmentioning

confidence: 99%

KinectAvatar: Fully Automatic Body Capture Using a Single Kinect

Cui¹,

Chang²,

Noll³

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We present a novel scanning system for capturing a full 3D human body model using just a single depth camera and no auxiliary equipment. We claim that data captured from a single Kinect is sufficient to produce a good quality full 3D human model. In this setting, the challenges we face are the sensor's low resolution with random noise and the subject's non-rigid movement when capturing the data. To overcome these challenges, we develop an improved superresolution algorithm that takes color constraints into account. We then align the super-resolved scans using a combination of automatic rigid and non-rigid registration. As the system is of low price and obtains impressive results in several minutes, full 3D human body scanning technology can now become more accessible to everyday users at home.

show abstract

Section: Related Workmentioning

confidence: 99%

KinectAvatar: Fully Automatic Body Capture Using a Single Kinect

Cui¹,

Chang²,

Noll³

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…A number of approaches have been proposed for temporal alignment of mesh sequences based on sequential frame-to-frame surface tracking. These can be categorised into two methodologies: model-based approaches which assume a known non-rigid object class such as a person and align a prior model of the surface with successive frames (Carranza et al 2003;de Aguiar et al 2008;Starck and Hilton 2003;Vlasic et al 2008); and surfacetracking or scene flow approaches which do not assume prior knowledge of the surface structure allowing alignment of freeform non-rigid surfaces (Cagniart et al 2010a;Hilton 2005, 2007a;Tevs et al 2011;Vedula et al 2005;Wand et al 2009). …”

Section: Sequential Non-rigid Alignmentmentioning

confidence: 99%

“…Specific approaches have also been introduced for extension of optical flow to tracking of 3D facial motion from reconstructed surface sequences (Zhang et al 2004;Bradley et al 2010). Tevs et al (2011) propose an approach referred to as animation cartography where a consistent map for surface patches is constructed based on tracking of sparse landmarks. Dense mapping and correspondence of the surface is constructed assuming isometric deformation.…”

Section: Free-form Sequential Alignmentmentioning

confidence: 99%

“…Research has focused on the problem of extracting a temporally consistent mesh representations over sequences by non-rigid surface tracking using either modelbased (de Aguiar et al 2008;Starck and Hilton 2003;Vlasic et al 2008) or model-free Bronstein et al 2007;Cagniart et al 2010b;Starck and Hilton 2007a;Tevs et al 2011;Tung and Matsuyama 2010;Zeng et al 2010) approaches. These single sequence alignment approaches commonly employ a sequential frame-toframe registration assuming relatively small non-rigid deformation between consecutive frames.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Global Non-rigid Alignment of Surface Sequences

et al. 2012

View full text Add to dashboard Cite

This paper presents a general approach based on the shape similarity tree for non-sequential alignment across databases of multiple unstructured mesh sequences from non-rigid surface capture. The optimal shape similarity tree for non-rigid alignment is defined as the minimum spanning tree in shape similarity space. Non-sequential alignment based on the shape similarity tree minimises the total non-rigid deformation required to register all frames in a database into a consistent mesh structure with surfaces in correspondence. This allows alignment across multiple sequences of different motions, reduces drift in sequential alignment and is robust to rapid non-rigid motion. Evaluation is performed on three benchmark databases of 3D mesh sequences with a variety of complex human and cloth motion. Comparison with sequential alignment demonstrates reduced errors due to drift and improved robustness to large non-rigid deformation, together with global alignment across multiple sequences which is not possible with previous sequential approaches.

show abstract

“…However, these approaches assume a reconstruction of the full non-rigid object surface at each time frame and do not easily extend to 4D alignment of partial surface reconstructions or depth maps. The wide-spread availability of low-cost depth sensors has motivated the development of methods for temporal correspondence or alignment and 4D modelling from partial dynamic surface observations [8,9,10,11]. Scene flow techniques [12,13] typically estimate the pairwise surface or volume correspondence between reconstructions at successive frames but do not extend to 4D alignment or correspondence across complete sequences due to drift and failure for rapid and complex motion.…”

Section: Introductionmentioning

confidence: 99%

4D Match Trees for Non-rigid Surface Alignment

Mustafa

Kim

Hilton

2016

Computer Vision – ECCV 2016

View full text Add to dashboard Cite

Abstract. This paper presents a method for dense 4D temporal alignment of partial reconstructions of non-rigid surfaces observed from single or multiple moving cameras of complex scenes. 4D Match Trees are introduced for robust global alignment of non-rigid shape based on the similarity between images across sequences and views. Wide-timeframe sparse correspondence between arbitrary pairs of images is established using a segmentation-based feature detector (SFD) which is demonstrated to give improved matching of non-rigid shape. Sparse SFD correspondence allows the similarity between any pair of image frames to be estimated for moving cameras and multiple views. This enables the 4D Match Tree to be constructed which minimises the observed change in non-rigid shape for global alignment across all images. Dense 4D temporal correspondence across all frames is then estimated by traversing the 4D Match tree using optical flow initialised from the sparse feature matches. The approach is evaluated on single and multiple view images sequences for alignment of partial surface reconstructions of dynamic objects in complex indoor and outdoor scenes to obtain a temporally consistent 4D representation. Comparison to previous 2D and 3D scene flow demonstrates that 4D Match Trees achieve reduced errors due to drift and improved robustness to large non-rigid deformations.

show abstract

Animation cartography—intrinsic reconstruction of shape and motion

Cited by 78 publications

References 48 publications

KinectAvatar: Fully Automatic Body Capture Using a Single Kinect

KinectAvatar: Fully Automatic Body Capture Using a Single Kinect

Global Non-rigid Alignment of Surface Sequences

4D Match Trees for Non-rigid Surface Alignment

Contact Info

Product

Resources

About