Template-free monocular reconstruction of deformable surfaces

Varol, Aydin; Salzmann, Mathieu; Tola, Engin; Fua, Pascal

doi:10.1109/iccv.2009.5459403

Cited by 86 publications

(86 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In work developed in parallel to ours, and similar in spirit, Taylor et al [21] have also proposed a piecewise approach to solve the NRSfM problem using locally-rigid SfM to reconstruct a soup of rigid triangles. Our approach is closely related to the work of Varol et al [22] who also propose a model free piecewise reconstruction of non-rigid surfaces but based on planar patches with overlapping points. An initial planar reconstruction is estimated for each patch from the homographies estimated from pairwise correspondences.…”

Section: Related Workmentioning

confidence: 95%

See 1 more Smart Citation

Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences

Fayad

Agapito

Bue

2010

Computer Vision – ECCV 2010

View full text Add to dashboard Cite

Abstract. In this paper we present a new method for the 3D reconstruction of highly deforming surfaces (for instance a flag waving in the wind) viewed by a single orthographic camera. We assume that the surface is described by a set of feature points which are tracked along an image sequence. Most non-rigid structure from motion algorithms assume a global deformation model where a rigid mean shape component accounts for most of the motion and the deformation modes are small deviations from it. However, in the case of strongly deforming objects, the deformations become more complex and a global model will often fail to explain the intricate deformations which are no longer small linear deviations from a strong mean component. Our proposed algorithm divides the surface into overlapping patches, reconstructs each of these patches individually using a quadratic deformation model and finally registers them imposing the constraint that points shared by patches must correspond to the same 3D points in space. We show good results on challenging motion capture and real video sequences with strong deformations where global methods fail to achieve good reconstructions.

show abstract

Section: Related Workmentioning

confidence: 95%

“…Piecewise quadratic reconstruction of real sequence: Figure 7 (top and middle rows) shows the reconstruction of a real sequence showing a paper bending [22]. Reconstructed points are represented in different colours showing the 36 patches used in the reconstruction.…”

Section: Gt I ||/||Xmentioning

confidence: 99%

Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences

Fayad

Agapito

Bue

2010

Computer Vision – ECCV 2010

View full text Add to dashboard Cite

show abstract

“…One of the unsolved problems therefore concerns the use of a more advanced model of the laparoscopic environment's Bidirectional Reflectance Distribution Function (BRDF). An interesting avenue for future research is applying template-free DSfM in the context of laparoscopy (Taylor et al, 2010;Russell et al, 2011;Collins and Bartoli, 2010;Varol et al, 2009) and studying its combination with SfS (Moreno-Noguer et al, 2010).…”

Section: Discussionmentioning

confidence: 99%

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

Maier-Hein

Mountney

Bartoli

et al. 2013

Medical Image Analysis

242

130

View full text Add to dashboard Cite

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon's navigation capabilites by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D optical imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions.

show abstract

“…Yet, while low-rank meth-ods can effectively encode global deformations, they cannot, in general, handle non-linear motion patterns and strong local deformations. Piecewise strategies (Chhatkuli et al 2014;Russell et al 2011;Taylor et al 2010;Varol et al 2009) allow recovering larger deformations, although their performance highly depends on having overlapping features in neighboring patches, or require large number of correspondences to enforce local rigidity constraints (Chhatkuli et al 2014;Taylor et al 2010;Varol et al 2009), which can be hard to obtain in practice. In any event, these previous approaches batch process all frames of the sequence at once, after video capture, preventing them from being used online and in real-time applications, where NRSfM may have an enormous potential.…”

mentioning

confidence: 99%

Combining Local-Physical and Global-Statistical Models for Sequential Deformable Shape from Motion

Agudo

Moreno-Noguer

2016

Int J Comput Vis

View full text Add to dashboard Cite

In this paper, we simultaneously estimate camera pose and non-rigid 3D shape from a monocular video, using a sequential solution that combines local and global representations. We model the object as an ensemble of particles, each ruled by the linear equation of the Newton's second law of motion. This dynamic model is incorporated into a bundle adjustment framework, in combination with simple regularization components that ensure temporal and spatial consistency. The resulting approach allows to sequentially estimate shape and camera poses, while progressively learning a global low-rank model of the shape that is fed back into the optimization scheme, introducing thus, global constraints. The overall combination of local (physical) and global (statistical) constraints yields a solution that is both efficient and robust to several artifacts such as noisy and missing data or sudden camera motions, without requiring any training data at all. Validation is done in a variety of real application domains, including articulated and non-rigid motion, both for continuous and discontinuous shapes. Our on-line methodology yields significantly more accurate reconstructions than competing sequential approaches, being even comparable to the more computationally demanding batch methods.

show abstract

Template-free monocular reconstruction of deformable surfaces

Abstract: Abstract

Cited by 86 publications

References 25 publications

Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences

Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

Combining Local-Physical and Global-Statistical Models for Sequential Deformable Shape from Motion

Contact Info

Product

Resources

About