Organic Priors in Non-Rigid Structure from Motion

Abstract

This paper advocates the use of organic priors in classical non-rigid structure from motion (NRSf M). By organic priors, we mean invaluable intermediate prior information intrinsic to the NRSf M matrix factorization theory. It is shown that such priors reside in the factorized matrices, and quite surprisingly, existing methods generally disregard them. The paper's main contribution is to put forward a simple, methodical, and practical method that can effectively exploit such organic priors to solve NRSf M. The proposed method does not make assumptions other than the popular one on the low-rank shape and offers a reliable solution to NRSf M under orthographic projection. Our work reveals that the accessibility of organic priors is independent of the camera motion and shape deformation type. Besides that, the paper provides insights into the NRSf M factorization-both in terms of shape and motion-and is the first approach to show the benefit of single rotation averaging for NRSf M. Furthermore, we outline how to effectively recover motion and non-rigid 3D shape using the proposed organic prior based approach and demonstrate results that outperform prior-free NRSf M performance by a significant margin. Finally, we present the benefits of our method via extensive experiments and evaluations on several benchmark datasets.

Deep Spatial-Temporal Neural Network for Dense Non-Rigid Structure from Motion

“…The widely usage of synthetic faces allows us to compare DST-NRSfM to more methods. To more intuitively understand the efficacy of our approach, we compared the experimental outcomes of DST-NRSfM with classical sparse NRSfM methods, such as metric projections (MP) [55], complementary rank-3 spaces (CSF2) [17], block-matrix-method (BMM) [20], and organic priors based approach (OP) [3], traditional dense NRSfM methods, such as variational approach (VA) [7], dense spatio-temporal approach (DSTA) [25], CMDR [52,53], Grassmannian manifold (GM) [8], jumping manifolds (JM) [29], SMSR [51], and probabilistic point trajectory approach (PPTA) [14], and the latest neural-based dense NRSfM approaches, such as N-NRSfM [9], and RONN [54]. Table 4 presents the final comparative experimental results, where OP is the newest method that solves the dense NRSfM problem by extending the sparse approach to the dense domain; however, the accuracy of our proposed framework remained nearly twice as accuracy.…”

“…A template-based approach for reconstructing dense surfaces was proposed by Russell et al [28] in 2012; however, in this approach, an appropriate 3D template must first be selected. The initial method for sparse reconstruction [1][2][3][4][5] was expanded to address the dense NRSfM problem because of the proliferation of NRSfM problem-solving techniques; however, the outcome remains subpar. Currently, an increasing number of studies are focused on solving the dense NRSfM problem; these include the following: (1) Agudo et al [6] proposed modeling time-varying shapes using a probabilistic linear subspace of modal shapes obtained from continuum mechanics.…”

“…Dense NRSfM is more complex and advances more slowly because it must reconstruct thousands of times more feature points compared with the sparse NRSFM. Most algorithms that address the dense NRSfM issue [1][2][3][4][5] extend the sparse reconstruction approach, and generally achieve unsatisfactory results. Recent developments have been made in dense NRSfM-specific algorithms [6][7][8][9].…”