Perspective Nonrigid Shape and Motion Recovery

Abstract

Abstract. We present a closed form solution to the nonrigid shape and motion (NRSM) problem from point correspondences in multiple perspective uncalibrated views. Under the assumption that the nonrigid object deforms as a linear combination of K rigid shapes, we show that the NRSM problem can be viewed as a reconstruction problem from multiple projections from P 3K to P 2 . Therefore, one can linearly solve for the projection matrices by factorizing a multifocal tensor. However, this projective reconstruction in P 3K does not satisfy the constraints of the NRSM problem, because it is computed only up to a projective transformation in P 3K . Our key contribution is to show that, by exploiting algebraic dependencies among the entries of the projection matrices, one can upgrade the projective reconstruction to determine the affine configuration of the points in R 3 , and the motion of the camera relative to their centroid. Moreover, if K ≥ 2, then either by using calibrated cameras, or by assuming a camera with fixed internal parameters, it is possible to compute the Euclidean structure by a closed form method.

Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences

“…The approaches developed so far to deal with the ambiguities can be classified according to the amount of a priori knowledge that they require. This can range from the strongest assumption of a known model [11], to which the current measurements are fit, to the model-free approach championed by NRSfM methods [1,12,6,8,4,9] or intermediate assumptions such as the requirement of a reference image in which the shape is known a priori [13,14].…”

“…Other approaches focus on ensuring that the solution lies on the correct motion manifold where the metric constraints are exactly satisfied [15]. The linear basis shape model has also allowed the formulation of closed form solutions both for the affine [5] and perspective [4,9] cases. However, closed form solutions are known to be very sensitive to noise [3,7] and cannot deal with missing data.…”

“…Most NRSfM methods are based on the low rank basis shape model defined by Bregler et al [1] in which the deformable 3D shape is represented as a linear combination of a set of basis shapes with time varying coefficients. This model has allowed the development of a family of algorithms which have been a successful extension of classical rigid factorization algorithms to the non-rigid case [2,3,4,5,6,7,8,9]. However, so far, these algorithms have been demonstrated on simple sequences where the deformations are only small deviations from a rigid component.…”

Shape Function-Based Estimation of Deformation With Moving Cameras Attached to the Deforming Body

“…Fayad et al (2009) extend the method to a quadratic deformation model, which is able to cope with object bending, stretching, shearing, and twisting. Most of these approaches that utilize shape bases assume an orthographic camera projection model, while Hartley and Vidal (2008) propose an algorithm for a perspective camera.…”

Non-rigid structure from motion using quadratic deformation models

“…The most common assumption, made originally by Bregler et al [5] and later adopted by most NRSfM methods, is that the deformable 3D shape can be represented as a linear combination of rigid basis shapes, or modes of deformation, with time varying coefficients. This linear shape model has allowed the development of a number of algorithms which can be seen as extensions of Tomasi and Kanade's classical rigid factorization algorithm [2,3,4,6,9,13,16,18,19].…”