NRSfM using local rigidity

Abstract

In this paper we show that typical nonrigid structure can often be approximated well as locally rigid sub-structures in time and space. Specifically, we assume that: I) the structure can be approximated as rigid in a short local time window and 2) some point-pairs stay relatively rigid in space, maintaining a fixed distance between them during the sequence. First, we use the triangulation constraints in rigid SjM over a sliding time window to get an initial estimate of the nonrigid 3D structure. Then we automat ically identify relatively rigid point-pairs in this structure, and use their length-constancy simultaneously with trian gulation constraints to refine the structure estimate. Lo cal factorization inherently handles small camera motion, short sequences and significant natural occlusions grace fully, performing better than nonrigid factorization meth ods. We show more stable and accurate results as compared to the state-of-the art on even short sequences starting from 15 frames only, containing camera rotations as small as 2° and up to 50% contiguous missing data.

Reconstruction of Articulated Objects from a Moving Camera

“…We use the 'skin' dataset from [26], a motion capture dataset with groundtruth 3D positions, which was also used in [14] to compare to [38]. We rotate a virtual perspective camera (using 5 o perframe separation, similar to [28]) around the object, project the 3D vertex positions into the virtual cameras, and then use our method to recover the 3D coordinates. It is worth noting that, our method works directly on perspective images, while prior works [14,38] employ a simpler and less realistic orthographic camera model for the reconstruction.…”

Accurate 3D Reconstruction of Dynamic Scenes from Monocular Image Sequences with Severe Occlusions

“…This statistical constraint can be interpreted as a basic form of a shape prior, and reflects the assumption on the linearity of deformations. This setting is known to perform well for moderate deformations and many successor methods built upon the idea of metric space constraints [47,27,36,31]. In contrast, SPVA determines optimal basis shapes implicitly by penalizing nuclear norm of the shape matrix as proposed in [13].…”

“…NRSfM methods made significant advances during recent years in terms of the ability to reconstruct realistic nonrigid motion, especially for image sequences and motion capture data acquired in a controlled environment. Along with methods supporting an orthographic camera model [47,29,30,7,27,15,41,31,5], there are methods supporting a full perspective (in most of the cases calibrated) camera model [50,21,8,23,52,49,6,12], dense reconstructions [36,15,2], sequential processing [26,43,1,4,3] and compound scenes [37]. At the same time, NRSfM is a highly ill-posed inverse problem in the sense of Hadamard, i.e., the condition on the uniqueness of the solution is violated.…”

“…At the same time, NRSfM is a highly ill-posed inverse problem in the sense of Hadamard, i.e., the condition on the uniqueness of the solution is violated. In practice, a prior knowledge is required to disambiguate the solution space such as metric constraints [27], constraints on point trajectories [7,53,48,4], temporal consistency assumption [15], local rigidity assumption [44,31], soft inextensibility constraint [49,12,3], shape prior [14,42,41,43] or the assumption on a compliance with a physical deformation model [4,6].…”

Non-Rigid Structure Estimation in Trajectory Space from Monocular Vision

“…On this basis, Gotardo and Matinez [ 9 ] combined the shape method and trajectory method, which can further improve the reconstruction performance. Recently, Rehan et al [ 10 ] proposed a novel constraint in the form of local rigidity, which gave stable results in challenging realistic scenarios with small camera motions and shorter sequences. Minsk et al [ 11 ] introduced new constraints that were more effective for non-rigid structure estimation, which constrained the motion parameters so that the 3D shapes were most closely aligned to each other, making the rank constraints unnecessary.…”