Complex Non-rigid 3D Shape Recovery Using a Procrustean Normal Distribution Mixture Model

Cho, Jungchan; Lee, Minsik; Oh, Songhwai

doi:10.1007/s11263-015-0860-7

Cited by 21 publications

(12 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most closely related to the present paper are generic factorization approaches for recovering 3D non-rigid shapes from image sequences captured with a single camera [7,3,14,57,12], i.e., non-rigid structure from motion (NRSFM), and human pose recovery models based on known skeletons [20,43,47,30,21] or sparse representations [32,15,2,55,56]. Much of this work has been realized by assuming manually labeled 2D joint locations; however, there is some recent work that has used a 2D pose detector to automatically provide the input joints [40,48] or solved 2D and 3D pose estimation jointly [39,54].…”

Section: ……mentioning

confidence: 99%

Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video

Zhou

Zhu

Leonardos

et al. 2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

395

407

View full text Add to dashboard Cite

This paper addresses the challenge of 3D full-body human pose estimation from a monocular image sequence. Here, two cases are considered: (i) the image locations of the human joints are provided and (ii) the image locations of joints are unknown. In the former case, a novel approach is introduced that integrates a sparsity-driven 3D geometric prior and temporal smoothness. In the latter case, the former case is extended by treating the image locations of the joints as latent variables to take into account considerable uncertainties in 2D joint locations. A deep fully convolutional network is trained to predict the uncertainty maps of the 2D joint locations. The 3D pose estimates are realized via an Expectation-Maximization algorithm over the entire sequence, where it is shown that the 2D joint location uncertainties can be conveniently marginalized out during inference. Empirical evaluation on the Human3.6M dataset shows that the proposed approaches achieve greater 3D pose estimation accuracy over state-of-the-art baselines. Further, the proposed approach outperforms a publicly available 2D pose estimation baseline on the challenging PennAction dataset.

show abstract

Section: ……mentioning

confidence: 99%

Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video

Zhou

Zhu

Leonardos

et al. 2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

395

407

View full text Add to dashboard Cite

show abstract

“…Most closely related to our work are generic factorization approaches for recovering 3D non-rigid shapes from image sequences captured with a single camera [4], [47], [48], [49], [50], i.e., non-rigid structure from motion (NRSFM), and human pose recovery models based on known skeletons [2], [3], [51], [52], [53], [54] or sparse representations [5], [55], [56], [57], [58]. Much of this work has been realized by assuming manually labeled 2D joint locations; however, there is some recent work that has used a 2D pose detector to automatically provide the input joints [59], [60] or solved 2D and 3D pose estimation jointly [61], [12].…”

Section: Related Workmentioning

confidence: 99%

MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior

Zhou

Zhu

Pavlakos

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

185

View full text Add to dashboard Cite

Recovering 3D full-body human pose is a challenging problem with many applications. It has been successfully addressed by motion capture systems with body worn markers and multiple cameras. In this paper, we address the more challenging case of not only using a single camera but also not leveraging markers: going directly from 2D appearance to 3D geometry. Deep learning approaches have shown remarkable abilities to discriminatively learn 2D appearance features. The missing piece is how to integrate 2D, 3D and temporal information to recover 3D geometry and account for the uncertainties arising from the discriminative model. We introduce a novel approach that treats 2D joint locations as latent variables whose uncertainty distributions are given by a deep fully convolutional neural network. The unknown 3D poses are modeled by a sparse representation and the 3D parameter estimates are realized via an Expectation-Maximization algorithm, where it is shown that the 2D joint location uncertainties can be conveniently marginalized out during inference. Extensive evaluation on benchmark datasets shows that the proposed approach achieves greater accuracy over state-of-the-art baselines. Notably, the proposed approach does not require synchronized 2D-3D data for training and is applicable to "in-the-wild" images, which is demonstrated with the MPII dataset.

show abstract

“…This should in practice factor out the rigid transformations from the statistical distribution of deformations. The PND model has been then extended to deal with more complex deformations and longer sequences (Cho et al 2016).…”

Section: Statisticalmentioning

confidence: 99%

A Benchmark and Evaluation of Non-Rigid Structure from Motion

Jensen¹,

Doest²,

Aanæs³

et al. 2020

Int J Comput Vis

View full text Add to dashboard Cite

Non-rigid structure from motion (nrsfm), is a long standing and central problem in computer vision and its solution is necessary for obtaining 3D information from multiple images when the scene is dynamic. A main issue regarding the further development of this important computer vision topic, is the lack of high quality data sets. We here address this issue by presenting a data set created for this purpose, which is made publicly available, and considerably larger than the previous state of the art. To validate the applicability of this data set, and provide an investigation into the state of the art of nrsfm, including potential directions forward, we here present a benchmark and a scrupulous evaluation using this data set. This benchmark evaluates 18 different methods with available code that reasonably spans the state of the art in sparse nrsfm. This new public data set and evaluation protocol will provide benchmark tools for further development in this challenging field.

show abstract

Complex Non-rigid 3D Shape Recovery Using a Procrustean Normal Distribution Mixture Model

Cited by 21 publications

References 22 publications

Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video

Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video

MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior

A Benchmark and Evaluation of Non-Rigid Structure from Motion

Contact Info

Product

Resources

About