MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior

Zhou, Xiaowei; Zhu, Menglong; Pavlakos, Georgios; Leonardos, Spyridon; Derpanis, Konstantinos G.; Daniilidis, Kostas

doi:10.1109/tpami.2018.2816031

Cited by 185 publications

(105 citation statements)

References 69 publications

(154 reference statements)

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…Our network predicts the normalized locations of 3D joints. During testing, to calibrate the scale of the outputs, we require that the sum of length of all 3D bones is equal to that of a canonical skeleton as shown in [41,75,78]. Therefore, we follow the method in [75] for calibration.…”

Section: Datasets and Evaluation Protocolsmentioning

confidence: 99%

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Zhao

Peng

Tian

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

517

398

View full text Add to dashboard Cite

In this paper, we study the problem of learning Graph Convolutional Networks (GCNs) for regression. Current architectures of GCNs are limited to the small receptive field of convolution filters and shared transformation matrix for each node. To address these limitations, we propose Semantic Graph Convolutional Networks (SemGCN), a novel neural network architecture that operates on regression tasks with graph-structured data. SemGCN learns to capture semantic information such as local and global node relationships, which is not explicitly represented in the graph. These semantic relationships can be learned through end-to-end training from the ground truth without additional supervision or hand-crafted rules. We further investigate applying SemGCN to 3D human pose regression. Our formulation is intuitive and sufficient since both 2D and 3D human poses can be represented as a structured graph encoding the relationships between joints in the skeleton of a human body. We carry out comprehensive studies to validate our method. The results prove that SemGCN outperforms state of the art while using 90% fewer parameters. The code can be found at https://github.com/garyzhao/SemGCN .

show abstract

Section: Datasets and Evaluation Protocolsmentioning

confidence: 99%

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Zhao

Peng

Tian

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

517

398

View full text Add to dashboard Cite

show abstract

“…There is rich recent literature on 3D pose estimation in the form of a simplistic body skeleton, e.g., [3,19,22,24,25,29,30,34,35,38,40,41,42,50,51]. However, in this Section, we focus on the more relevant works recovering the full shape and pose of the human body.…”

Section: Related Workmentioning

confidence: 99%

Convolutional Mesh Regression for Single-Image Human Shape Reconstruction

Kolotouros

Pavlakos

Daniilidis

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

549

595

View full text Add to dashboard Cite

This paper addresses the problem of 3D human pose and shape estimation from a single image. Previous approaches consider a parametric model of the human body, SMPL, and attempt to regress the model parameters that give rise to a mesh consistent with image evidence. This parameter regression has been a very challenging task, with modelbased approaches underperforming compared to nonparametric solutions in terms of pose estimation. In our work, we propose to relax this heavy reliance on the model's parameter space. We still retain the topology of the SMPL template mesh, but instead of predicting model parameters, we directly regress the 3D location of the mesh vertices. This is a heavy task for a typical network, but our key insight is that the regression becomes significantly easier using a Graph-CNN. This architecture allows us to explicitly encode the template mesh structure within the network and leverage the spatial locality the mesh has to offer. Image-based features are attached to the mesh vertices and the Graph-CNN is responsible to process them on the mesh structure, while the regression target for each vertex is its 3D location. Having recovered the complete 3D geometry of the mesh, if we still require a specific model parametrization, this can be reliably regressed from the vertices locations. We demonstrate the flexibility and the effectiveness of our proposed graphbased mesh regression by attaching different types of features on the mesh vertices. In all cases, we outperform the comparable baselines relying on model parameter regression, while we also achieve state-of-the-art results among model-based pose estimation approaches. 1 arXiv:1905.03244v1 [cs.CV] 8 May 2019• We achieve state-of-the-art results among model-based pose estimation approaches.

show abstract

“…Weakly Supervised: Approaches such as [3,10,44,53,54,55] do not explicitly use paired 2D-3D correspondences, but use unpaired 3D data to learn priors on shape (3D basis) or pose (articulation priors). For example, Zhou et al [54] use a 3D pose dictionary to learn pose priors and Brau et al [3] employ an independently trained network that learns a prior distribution over 3D poses (kinematic and self-intersection priors).…”

Section: Related Workmentioning

confidence: 99%

Unsupervised 3D Pose Estimation With Geometric Self-Supervision

Chen

Tyagi

Agrawal

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

204

165

View full text Add to dashboard Cite

We present an unsupervised learning approach to recover 3D human pose from 2D skeletal joints extracted from a single image. Our method does not require any multiview image data, 3D skeletons, correspondences between 2D-3D points, or use previously learned 3D priors during training. A lifting network accepts 2D landmarks as inputs and generates a corresponding 3D skeleton estimate. During training, the recovered 3D skeleton is reprojected on random camera viewpoints to generate new 'synthetic' 2D poses. By lifting the synthetic 2D poses back to 3D and re-projecting them in the original camera view, we can define self-consistency loss both in 3D and in 2D. The training can thus be self supervised by exploiting the geometric selfconsistency of the lift-reproject-lift process. We show that self-consistency alone is not sufficient to generate realistic skeletons, however adding a 2D pose discriminator enables the lifter to output valid 3D poses. Additionally, to learn from 2D poses 'in the wild', we train an unsupervised 2D domain adapter network to allow for an expansion of 2D data. This improves results and demonstrates the usefulness of 2D pose data for unsupervised 3D lifting. Results on Human3.6M dataset for 3D human pose estimation demonstrate that our approach improves upon the previous unsupervised methods by 30% and outperforms many weakly supervised approaches that explicitly use 3D data.

show abstract

MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior

Cited by 185 publications

References 69 publications

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Convolutional Mesh Regression for Single-Image Human Shape Reconstruction

Unsupervised 3D Pose Estimation With Geometric Self-Supervision

Contact Info

Product

Resources

About