Optimal shape from motion estimation with missing and degenerate data

Abstract

Reconstructing a 3D scene from a moving camera is one of the most important issues in the field of computer vision. In this scenario, not all points are known in all images (e.g. due to occlusion), thus generating missing data. The state of the art handles the missing points in this context by enforcing rank constraints on the point track matrix. However, quite frequently, close up views tend to capture planar surfaces producing degenerate data. If one single frame is degenerate, the whole sequence will produce high errors on the shape reconstruction, even though the observation matrix verifies the rank 4 constraint. In this paper, we propose to solve the structure from motion problem with degenerate data, introducing a new factorization algorithm that imposes the full scaled orthographic model in one single optimization procedure. By imposing all model constraints, a unique (correct) 3D shape is estimated regardless of the data degeneracies. Experiments show that remarkably good reconstructions are obtained with an approximate models such as orthography.

Combining Local-Physical and Global-Statistical Models for Sequential Deformable Shape from Motion

“…For this purpose, we follow (Agudo et al 2014a,b;Paladini et al 2010), and assume that the sequence contains a few initial frames where the object does not undergo large deformations. We use a standard practice done in NRSfM, that is running a rigid factorization algorithm (Marques and Costeira 2008) on these first frames to obtain a shape and pose estimate. Let us denote by s 0 the shape at rest.…”

A Linear Solution to 1-Dimensional Subspace Fitting under Incomplete Data

“…We obtain a projective reconstruction by iterating the following three steps: estimate t by Eq. (16), estimate all unknown observationsx ij by Eq. (17), and optimize the projective depths λ ij by Eq.…”

“…Buchanon and Fitzgibbon [4] proposed to improve convergence speed by combining a Gauss-Newton approach [17,18] with the method of [11]. Recently, it was proposed to integrate problem-specific constraints into EMschemes [16,1].…”

“…Conversely, EM-schemes [23,22,8,19,16,1] or EM-like schemes [11] seem to converge more robustly, yet can require tremendously much time [4]. Approaches generalizing some estimate obtained from some part of the data to the complete data are highly susceptible to noise, larger amounts of missing data and degenerate camera configurations [20,13].…”

Trajectory reconstruction for affine structure-from-motion by global and local constraints

“…Marquez and Costeira [13] introduced a non-incremental method for estimating the missing data. They iteratively estimate the unobserved feature points.…”