Soft Inextensibility Constraints for Template-Free Non-rigid Reconstruction

Abstract

Abstract. In this paper, we exploit an inextensibility prior as a way to better constrain the highly ambiguous problem of non-rigid reconstruction from monocular views. While this widely applicable prior has been used before combined with the strong assumption of a known 3D-template, our work achieves template-free reconstruction using only inextensibility constraints. We show how to formulate an energy function that includes soft inextensibility constraints and rely on existing discrete optimisation methods to minimise it. Our method has all of the following advantages: (i) it can be applied to two tasks that have been so far considered independently -template based reconstruction and non-rigid structure from motion -producing comparable or better results than the state-of-the art methods; (ii) it can perform template-free reconstruction from as few as two images; and (iii) it does not require post-processing stitching or surface smoothing.

Template-Free 3D Reconstruction of Poorly-Textured Nonrigid Surfaces

“…Fusion moves [32,33] is an optimization technique for graphical models, which can handle continuous variables by iteratively solving a discrete problem. In the context of deformable surfaces, this approach was employed by [19] to address the well-textured template-based and template-free scenarios. Cast to our problem, the fusion moves algorithm works in the following manner: Given the current solution at iteration t, (U, D, L) t , and a proposal, (U, D, L) p , the fusion moves algorithm combines the solution and proposal into a new solution (U, D, L) t+1 while ensuring that the energy of the new solution is at least as low as that of both the current solution and the proposal.…”

“…In this section, we briefly review the literature on monocular 3D reconstruction of deformable surfaces, i.e., template-based methods [1][2][3][4][5][6][7][8][9][10][11] and NRSfM [14][15][16][17][18][19]. In both cases, we focus the discussion on the methods that are the most relevant to our work.…”

“…In [20], isometry and infinitesimal planarity are employed, and reconstruction is achieved by integrating the normals obtained from the decomposition of local 2D homographies. In [19], isometry is encoded by encouraging the distance between neighboring feature points to remain constant over the entire sequence. Similarly to our approach, this method relies on a fusion moves optimization strategy, which makes it more robust to local minima than a gradient-based approach.…”

“…By contrast, NRSfM techniques [14][15][16][17][18][19][20] do not require knowing the shape of the object a priori. Instead, they make use of a video sequence as input, and estimate the shape of the surface in all the frames of this sequence.…”

Non-Rigid Shape-from-Motion for Isometric Surfaces using Infinitesimal Planarity

“…[3] uses the orthographic camera to recover the shape's normal locally; they suffer from local two-fold ambiguities and significantly degrade for shorter focal lengths. [5] recently solved the same problem for an orthographic and perspective camera. [4] specifically addresses the case of piecewise planar surfaces; it uses the perspective camera but still has patch-wise two-fold unresolved ambiguities induced by the processing of image pairs.…”