Provably-convergent iterative methods for projective structure from motion

Abstract

Astract: The estimation of the projective structure of a scene from image correspondences can be formulated as the minimization of the mean-squared distance between predicted and observed image points with respect to the projection matrices, the scene point positions, and their depths. Since these unknowns are not independent, constraints must be chosen to ensure that the optimization process is well posed. This paper examines three plausible choices, and shows that the first one leads to the Sturm-Triggs projective factorization algorithm, while the other two lead to new provably-convergent approaches. Experiments with synthetic and real data are used to compare the proposed techniques to the Sturm-Triggs algorithm and bundle adjustment. IntroductionLet us consider n fixed points P I , .., P,, observed by m perspective cameras. Given some fixed world coordinate system, we can write for i = 1, .., m and j = 1, ..,n, where p i j = (tiij,vij, l)T and zij denote respectively the (homogeneous) coordinate vector of the projection of Pj into image number i, expressed in the corresponding camera's coordinate frame and the depth of Pj relative to that frame, Mi is the 3 x 4 projection matrix associated with this camera in the world coordinate frame, and Pj is the homogeneous coordinate vector of the point Pj in that frame.We address the problem of reconstructing both the matrices Mi (i = l , ..,m) and the vectors Pj ( j = l, ..,TI) from the image correspondences p . . . Of course, zij is also unknown, but its value is not independent of the values of M i and Pi: indeed zij = mi3 . Pj , where m$ denotes the third row of the matrix M i . Faugeras [3] and Hartley et al. [7] have shown that when the internal parameters of the cameras are unknown, the cam-*3 'This work waa done while Y. Omori was visiting the Beckman Institute. He is now with the era motion and the scene structure can only be reconstructed up t o an arbitrary projective transformation

Perspective Nonrigid Shape and Motion Recovery

“…While factorization methods such as [9,18,8] are commonly used in affine reconstruction problems involving affine or orthographic cameras, they are not so useful for reconstruction from perspective cameras, since they require iterative estimation of the depth values [11,17]. For such problems an alternative is to use tensor-based methods.…”

“…As shown in [12], when the depths are known, the shape coefficients and shape basis may be computed from the factorization of W using a factorization technique similar to that in [8] for affine cameras. In [12], they solve the perspective reconstruction problem by alternately solving for the depths and the shape and motion parameters, in a similar way to [17]. In this paper, we seek an alternative purely algebraic solution to the problem that does not rely on any iterative optimization.…”

“…[9,11]. In fact, one can solve the SfM problem by alternating between the estimation of the depths, and the estimation of motion and structure [17], though care must be taken to avoid converging to trivial solutions [15].…”

Element-Wise Factorization for N-View Projective Reconstruction

“…balancing) will always converge to the trivial solution; (2) paper [10]'s provably-convergent iteration method will generally converge to a useless solution; (3) applying both row-wise and column-wise normalization may possibly run into unstable state during iteration. The authors also provided a remedy, i.e.…”

Robust Factorization Methods Using a Gaussian/Uniform Mixture Model

“…The generalization of affine factorization to deal with perspective implies the estimation of depth values associated with each reconstructed point. This is generally performed iteratively (Sturm and Triggs 1996;Christy and Horaud 1996;Mahamud and Hebert 2000;Mahamud et al 2001;Miyagawa and Arakawa 2006;Oliensis and Hartley 2007). It is not yet clear at all how to combine iterative robust methods with iterative projective/perspective factorization methods.…”