Laplacian Meshes for Monocular 3D Shape Recovery

Östlund, Jonas; Varol, Aydin; Ngo, Dat Tien; Fua, Pascal

doi:10.1007/978-3-642-33712-3_30

Cited by 45 publications

(41 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each one of these features can be found in isolation in other approaches but ours brings them all together in a unified framework. introduced in a conference paper [20] and is validated more thoroughly here.…”

Section: Introductionmentioning

confidence: 97%

Template-Based Monocular 3D Shape Recovery Using Laplacian Meshes

Ngo

Östlund

Fua

2016

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Abstract-We show that by extending the Laplacian formalism, which was first introduced in the Graphics community to regularize 3D meshes, we can turn the monocular 3D shape reconstruction of a deformable surface given correspondences with a reference image into a much better-posed problem. This allows us to quickly and reliably eliminate outliers by simply solving a linear least squares problem. This yields an initial 3D shape estimate, which is not necessarily accurate, but whose 2D projections are. The initial shape is then refined by a constrained optimization problem to output the final surface reconstruction. Our approach allows us to reduce the dimensionality of the surface reconstruction problem without sacrificing accuracy, thus allowing for real-time implementations.

show abstract

Section: Introductionmentioning

confidence: 97%

Template-Based Monocular 3D Shape Recovery Using Laplacian Meshes

Ngo

Östlund

Fua

2016

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…. ; v i Nc ], as demonstrated in a previous work [25]. There exists a linear relation x = Pc, where P is a constant parameterization matrix.…”

Section: Deformable Surface Trackingmentioning

confidence: 83%

“…However, its complex formulation is quite slow to optimize as reported in [7]. The work of Ostlund et al [25] takes inspiration from the Laplacian formalism presented in [35] and the rotation-invariant formulation of [36] to derive a rotation-invariant regularization term and a linear subspace parameterization of mesh vertices with respect to some control vertices. This technique leads to the first real-time 3-D deformable surface tracking system as reported in [24], which can run at 8-10 frames per second (FPS) on a MacBook Pro 2014 laptop.…”

Section: Deformable Surface Trackingmentioning

confidence: 99%

“…3. The works in [25,24] formulate the inequality constraints C (Pc) ≤ 0 as equality constraints with additional slack variables whose norm is penalized to prevent lengths from becoming too small and the solution from shrinking to the origin. They solve a complex optimization problem involving extra variables and hard constraints: min c,s…”

Section: Surface Reconstruction By Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Live Texturing of Augmented Reality Characters from Colored Drawings

Magnenat

Ngo

Zünd

et al. 2015

IEEE Trans. Visual. Comput. Graphics

Self Cite

View full text Add to dashboard Cite

Abstract-Coloring books capture the imagination of children and provide them with one of their earliest opportunities for creative expression. However, given the proliferation and popularity of digital devices, real-world activities like coloring can seem unexciting, and children become less engaged in them. Augmented reality holds unique potential to impact this situation by providing a bridge between real-world activities and digital enhancements. In this paper, we present an augmented reality coloring book App in which children color characters in a printed coloring book and inspect their work using a mobile device. The drawing is detected and tracked, and the video stream is augmented with an animated 3-D version of the character that is textured according to the child's coloring. This is possible thanks to several novel technical contributions. We present a texturing process that applies the captured texture from a 2-D colored drawing to both the visible and occluded regions of a 3-D character in real time. We develop a deformable surface tracking method designed for colored drawings that uses a new outlier rejection algorithm for real-time tracking and surface deformation recovery. We present a content creation pipeline to efficiently create the 2-D and 3-D content. And, finally, we validate our work with two user studies that examine the quality of our texturing algorithm and the overall App experience.

show abstract

“…Template-based reconstruction is an alternative method which usually relies on a known reference frame and works well especially for inextensible surfaces. Since this is still an ill-conditioned problem [21], the most commonly used constraints in the reconstruction involve preserving Geodesic distances as the surface deforms and thus regularise the problem by solving either convex optimisation problem [7,29] or in closed-form sets of quadratic equations [19,30]. The existing 3D reconstruction technologies have been successfully used in many different areas, ranging from medical imaging and biometrics to computer gaming and film production.…”

Section: Introductionmentioning

confidence: 99%

Robust deformable shape reconstruction from monocular video with manifold forests

Tao

Matuszewski²

2016

Machine Vision and Applications

View full text Add to dashboard Cite

Existing approaches to recover structure of 3D deformable objects and camera motion parameters from an uncalibrated images assume the object's shape could be modelled well by a linear subspace. These methods have been proven effective and well suited when the deformations are relatively small, but fail to reconstruct the objects with relatively large deformations. This paper describes a novel approach for 3D non-rigid shape reconstruction, based on manifold decision forest technique. The use of this technique can be justified by noting that a specific type of shape variations might be governed by only a small number of parameters, and therefore can be well represented in a lowdimensional manifold. The key contributions of this work are the use of random decision forests for the shape manifold learning and robust metric for calculation of the re-projection error. The learned manifold defines constraints imposed on the reconstructed shapes. Due to a nonlinear structure of the learned manifold, this approach is more suitable to deal with large and complex object deformations when compared to the linear constraints. The robust metric is applied to reduce the effect of measurement outliers on the quality of the reconstruction. In many practical applications outliers cannot be completely removed and therefore the use of robust techniques is of particular practical interest. The proposed method is validated on 2D points sequences projected from the 3D motion capture data for ground truth comparison and also on real 2D video sequences. Experiments show that the newly proposed method provides better performance compared to previously proposed ones, including the robustness with respect to measurement noise, missing measurements and outliers present in the data.

show abstract

Laplacian Meshes for Monocular 3D Shape Recovery

Cited by 45 publications

References 24 publications

Template-Based Monocular 3D Shape Recovery Using Laplacian Meshes

Template-Based Monocular 3D Shape Recovery Using Laplacian Meshes

Live Texturing of Augmented Reality Characters from Colored Drawings

Robust deformable shape reconstruction from monocular video with manifold forests

Contact Info

Product

Resources

About