Recursive non-rigid structure from motion with online learned shape prior

Tao, Lili; Mein, Stephen James; Quan, Wei; Matuszewski, Bogdan J.

doi:10.1016/j.cviu.2013.03.005

Cited by 12 publications

(22 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Very recently, (Agudo et al 2014a,b) presented the first approach to reconstruct both sparse and dense 3D shapes in a sequential manner, relying on a linear subspace of basis shapes computed by modal analysis. However, despite being very promising, these methods are only valid to handle smoothly deforming objects, as is the case of (Paladini et al 2010;Tao et al 2013), and cannot be applied to articulated motion (Agudo et al 2012(Agudo et al , 2014b(Agudo et al , 2016). …”

Section: Related Workmentioning

confidence: 99%

“…Specifically, Paladini et al (2010) proposed a 3D-implicit low-rank model to encode the time-varying shape, estimating the remaining model parameters by BA over a temporal sliding window. Based on a similar framework, Tao et al (2013) proposed an incremental principal component analysis to recursively update the low-rank model. Linear elasticity by means of finite element models was introduced into an extended Kalman filter to encode extensible deformations in real time (Agudo et al 2012), even computing the full camera trajectory (Agudo et al 2016).…”

Section: Related Workmentioning

confidence: 99%

“…While sequential solutions exist for the rigid case (Newcome and Davison 2010;Lim et al 2011), sequential estimation of deformable objects based only on the measurements up to that moment remains a challenging and unsolved problem. There are just a few attempts along this direction (Agudo et al 2016(Agudo et al , 2014a(Agudo et al , 2012Paladini et al 2010;Tao et al 2013). Specifically, Paladini et al (2010) proposed a 3D-implicit low-rank model to encode the time-varying shape, estimating the remaining model parameters by BA over a temporal sliding window.…”

Section: Related Workmentioning

confidence: 99%

“…For batch methods (Dai et al 2012;Del Bue et al 2006;Garg et al 2013;Torresani et al 2008), these low-rank models are learned after processing all frames of the sequence at once. For sequential ones (Paladini et al 2010;Tao et al 2013), the lowrank model is incrementally learned.…”

Section: Global Statistical Modelmentioning

confidence: 99%

“…In any event, these previous approaches batch process all frames of the sequence at once, after video capture, preventing them from being used online and in real-time applications, where NRSfM may have an enormous potential. This has been recently addressed in (Agudo et al 2014a(Agudo et al , 2012Paladini et al 2010;Tao et al 2013), which, however, still focus on global models only valid for relatively small deformations (Paladini et al 2010;Tao et al 2013) or continuous warps (Agudo et al 2014a(Agudo et al , 2012). …”

mentioning

confidence: 99%

See 4 more Smart Citations

Combining Local-Physical and Global-Statistical Models for Sequential Deformable Shape from Motion

Agudo

Moreno-Noguer

2016

Int J Comput Vis

View full text Add to dashboard Cite

In this paper, we simultaneously estimate camera pose and non-rigid 3D shape from a monocular video, using a sequential solution that combines local and global representations. We model the object as an ensemble of particles, each ruled by the linear equation of the Newton's second law of motion. This dynamic model is incorporated into a bundle adjustment framework, in combination with simple regularization components that ensure temporal and spatial consistency. The resulting approach allows to sequentially estimate shape and camera poses, while progressively learning a global low-rank model of the shape that is fed back into the optimization scheme, introducing thus, global constraints. The overall combination of local (physical) and global (statistical) constraints yields a solution that is both efficient and robust to several artifacts such as noisy and missing data or sudden camera motions, without requiring any training data at all. Validation is done in a variety of real application domains, including articulated and non-rigid motion, both for continuous and discontinuous shapes. Our on-line methodology yields significantly more accurate reconstructions than competing sequential approaches, being even comparable to the more computationally demanding batch methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Global Statistical Modelmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Combining Local-Physical and Global-Statistical Models for Sequential Deformable Shape from Motion

Agudo

Moreno-Noguer

2016

Int J Comput Vis

View full text Add to dashboard Cite

show abstract

A BRMF-Based Model for Missing-Data Estimation of Image Sequence

Liu

Sun

Jing

et al. 2015

Intelligent Computing Theories and Methodologies

View full text Add to dashboard Cite

Robust deformable shape reconstruction from monocular video with manifold forests

Tao

Matuszewski²

2016

Machine Vision and Applications

Self Cite

View full text Add to dashboard Cite

Existing approaches to recover structure of 3D deformable objects and camera motion parameters from an uncalibrated images assume the object's shape could be modelled well by a linear subspace. These methods have been proven effective and well suited when the deformations are relatively small, but fail to reconstruct the objects with relatively large deformations. This paper describes a novel approach for 3D non-rigid shape reconstruction, based on manifold decision forest technique. The use of this technique can be justified by noting that a specific type of shape variations might be governed by only a small number of parameters, and therefore can be well represented in a lowdimensional manifold. The key contributions of this work are the use of random decision forests for the shape manifold learning and robust metric for calculation of the re-projection error. The learned manifold defines constraints imposed on the reconstructed shapes. Due to a nonlinear structure of the learned manifold, this approach is more suitable to deal with large and complex object deformations when compared to the linear constraints. The robust metric is applied to reduce the effect of measurement outliers on the quality of the reconstruction. In many practical applications outliers cannot be completely removed and therefore the use of robust techniques is of particular practical interest. The proposed method is validated on 2D points sequences projected from the 3D motion capture data for ground truth comparison and also on real 2D video sequences. Experiments show that the newly proposed method provides better performance compared to previously proposed ones, including the robustness with respect to measurement noise, missing measurements and outliers present in the data.

show abstract

Recursive non-rigid structure from motion with online learned shape prior

Cited by 12 publications

References 14 publications

Combining Local-Physical and Global-Statistical Models for Sequential Deformable Shape from Motion

Combining Local-Physical and Global-Statistical Models for Sequential Deformable Shape from Motion

A BRMF-Based Model for Missing-Data Estimation of Image Sequence

Robust deformable shape reconstruction from monocular video with manifold forests

Contact Info

Product

Resources

About