Dynamic surface matching by geodesic mapping for 3D animation transfer

Abstract

This paper presents a novel approach that achieves complete matching of 3D dynamic surfaces. Surfaces are captured from multi-view video data and represented by sequences of 3D manifold meshes in motion (3D videos). We propose to perform dense surface matching between 3D video frames using geodesic diffeomorphisms. Our algorithm uses a coarse-to-fine strategy to derive a robust correspondence map, then a probabilistic formulation is coupled with a voting scheme in order to obtain local unicity of matching candidates and a smooth mapping. The significant advantage of the proposed technique compared to existing approaches is that it does not rely on a color-based feature extraction process. Hence, our method does not lose accuracy in poorly textured regions and is not bounded to be used on video sequences of a unique subject. Therefore our complete surface mapping can be applied to: (1) texture transfer between surface models extracted from different sequences, (2) dense motion flow estimation in 3D video, and (3) motion transfer from a 3D video to an unanimated 3D model. Experiments are performed on challenging publicly available real-world datasets and show compelling results.

Global Non-rigid Alignment of Surface Sequences

“…The optimal shape similarity tree is defined as the minimum spanning tree in shape similarity space which minimises the total non-rigid deformation for alignment across all frames. Non-rigid alignment is performed by traversing the shape similarity tree with pair-wise alignment of meshes using any existing sequential non-rigid alignment technique (de Aguiar et al 2008;Budd and Hilton 2009;Cagniart et al 2010b;Tung and Matsuyama 2010). Alignment based on the shape similarity tree optimises the path for non-rigid alignment giving reduced drift and improved robustness over previous sequential approaches.…”

“…A number of similarity measures for mesh sequences taking into account both shape and motion have been investigated (Tung and Matsuyama 2010;Huang et al 2010). In this work we utilise the temporally filtered volumetric shape histogram as a measure of shape and non-rigid motion similarity which has been shown to give good performance on reconstructed mesh sequences of people (Huang et al 2010).…”

“…Research has focused on the problem of extracting a temporally consistent mesh representations over sequences by non-rigid surface tracking using either modelbased (de Aguiar et al 2008;Starck and Hilton 2003;Vlasic et al 2008) or model-free Bronstein et al 2007;Cagniart et al 2010b;Starck and Hilton 2007a;Tevs et al 2011;Tung and Matsuyama 2010;Zeng et al 2010) approaches. These single sequence alignment approaches commonly employ a sequential frame-toframe registration assuming relatively small non-rigid deformation between consecutive frames.…”

A 3D Shape Descriptor for Segmentation of Unstructured Meshes into Segment-Wise Coherent Mesh Series

“…Therefore geodesic coordinate is invariant against mesh deformations between frames if there are no global topology changes [14]. Our shape descriptor is a set of geodesic histograms {H geo n,m (t)} where m = {1, ..., n} denotes geodesic coordinate corresponding to a tip.…”

“…Here, we apply tracking-by-matching approach to acquire segment-wise coherent mesh series from each mesh interval. In particular, we used the geodesic mapping [14] since the geodesic coordinate required in [14] has already been computed for our shape descriptor.…”