Articulated mesh animation from multi-view silhouettes

Abstract

Details in mesh animations are difficult to generate but they have great impact on visual quality. In this work, we demonstrate a practical software system for capturing such details from multi-view video recordings. Given a stream of synchronized video images that record a human performance from multiple viewpoints and an articulated template of the performer, our system captures the motion of both the skeleton and the shape. The output mesh animation is enhanced with the details observed in the image silhouettes. For example, a performance in casual loose-fitting clothes will generate mesh animations with flowing garment motions. We accomplish this with a fast pose tracking method followed by nonrigid deformation of the template to fit the silhouettes. The entire process takes less than sixteen seconds per frame and requires no markers or texture cues. Captured meshes are in full correspondence making them readily usable for editing operations including texturing, deformation transfer, and deformation model learning.

Implicit B-Spline Surface Reconstruction

“…3 The comparisons are applied for both 2D and 3D data sets, which are either public [13], [30] or obtained through a 3D scanner (i.e., Konica Minolta Vivid 9i) or multi-camera platform [31]. The comparisons are based on the geometric accuracy as well as the CPU time and the reconstruction time.…”

“…Fig. 12 illustrates the accuracy of the blended IBS that is reconstructed from the Bouncing and Handstand data sets in [31]. In this figure the results of blended IBS are compared with the RBF surface reconstruction [14].…”

Non-rigid Registration Meets Surface Reconstruction

“…The proposed registration framework has been validated for different data sets, which are either public [34] or obtained through a multi-view camera environment. The interfaces are reconstructed by the partition of unity weighting [24] applied on the quadratic patches acquired by [4]; octrees of depths 6, 7 or 8 have been used for the representation.…”

“…Table 1 shows how our method outperforms in terms of error and number of iterations. The accumulated errors in the last three rows are easily calculated since the ground truth correspondences are provided by [34]. For the first two rows, we use the distance from every target point to the tangent plane of its closest source point; it can be better estimated by using curvature information though [25].…”

Colour Dynamic Photometric Stereo for Textured Surfaces

“…that project time-varying textures on the surface are rather difficult to apply on deformable surfaces. Passive techniques, such as conventional stereo or multicamera systems, are not biased by temporal variation, and they are more frequently applied for dynamic surfaces [1][2][3][4][5][6][7]. However, their accuracy is not sufficient to reconstruct small details, such as wrinkles and creases of garments.…”