Monocular 3D Human Pose Estimation by Predicting Depth on Joints

Nie, Bruce Xiaohan; Wei, Ping; Zhu, Song‐Chun

doi:10.1109/iccv.2017.373

Cited by 100 publications

(67 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…or resort to an approximate solution [28]. The approach [54] tries to directly regress the absolute depth from the cropped and scaled image regions which is a very ambiguous task. In contrast, our approach does not make any assumptions, nor does it try to solve any ambiguous task.…”

Section: Related Workmentioning

confidence: 99%

Hand Pose Estimation via Latent 2.5D Heatmap Regression

Iqbal

Molchanov

Breuel

et al. 2018

Lecture Notes in Computer Science

276

338

View full text Add to dashboard Cite

Estimating the 3D pose of a hand is an essential part of human-computer interaction. Estimating 3D pose using depth or multiview sensors has become easier with recent advances in computer vision, however, regressing pose from a single RGB image is much less straightforward. The main difficulty arises from the fact that 3D pose requires some form of depth estimates, which are ambiguous given only an RGB image. In this paper we propose a new method for 3D hand pose estimation from a monocular image through a novel 2.5D pose representation. Our new representation estimates pose up to a scaling factor, which can be estimated additionally if a prior of the hand size is given. We implicitly learn depth maps and heatmap distributions with a novel CNN architecture. Our system achieves the state-of-the-art estimation of 2D and 3D hand pose on several challenging datasets in presence of severe occlusions.

show abstract

Section: Related Workmentioning

confidence: 99%

Hand Pose Estimation via Latent 2.5D Heatmap Regression

Iqbal

Molchanov

Breuel

et al. 2018

Lecture Notes in Computer Science

276

338

View full text Add to dashboard Cite

show abstract

“…Fully Supervised: These include approaches such as [26,44,23] that use paired 2D-3D data comprised of ground truth 2D locations of joint landmarks and corresponding 3D ground truth for learning. For example, Martinez et al .…”

Section: Related Workmentioning

confidence: 99%

Can 3D Pose Be Learned from 2D Projections Alone?

Drover

Rohith

Chen

et al. 2019

Lecture Notes in Computer Science

102

View full text Add to dashboard Cite

3D pose estimation from a single image is a challenging task in computer vision. We present a weakly supervised approach to estimate 3D pose points, given only 2D pose landmarks. Our method does not require correspondences between 2D and 3D points to build explicit 3D priors. We utilize an adversarial framework to impose a prior on the 3D structure, learned solely from their random 2D projections. Given a set of 2D pose landmarks, the generator network hypothesizes their depths to obtain a 3D skeleton. We propose a novel Random Projection layer, which randomly projects the generated 3D skeleton and sends the resulting 2D pose to the discriminator. The discriminator improves by discriminating between the generated poses and pose samples from a real distribution of 2D poses. Training does not require correspondence between the 2D inputs to either the generator or the discriminator. We apply our approach to the task of 3D human pose estimation. Results on Human3.6M dataset demonstrates that our approach outperforms many previous supervised and weakly supervised approaches.

show abstract

“…These methods are LinKDE [5], Tekin et al [15], Li et al [36], Zhou et al [14], Zhou et al [4], Du et al [37], Sanzari et al [34], Yasin et al [17], and Bogo et al [24]. Moreover, we compare other competing methods, i.e., Moreno-Noguer et al [38], Tome et al [20], Chen et al [16], Pavlakos et al [19], Zhou et al [23], Bruce et al [51], Tekin et al [50] and our conference version, i.e., Lin et al [21]. For those compared methods (i.e., [4], [5], [15], [16], [17], [24], [34], [36], [37], [38], [51]) whose source codes are not publicly available, we directly obtain their results from their published papers.…”

Section: Comparisons With Existing Methodsmentioning

confidence: 99%

“…Moreover, we compare other competing methods, i.e., Moreno-Noguer et al [38], Tome et al [20], Chen et al [16], Pavlakos et al [19], Zhou et al [23], Bruce et al [51], Tekin et al [50] and our conference version, i.e., Lin et al [21]. For those compared methods (i.e., [4], [5], [15], [16], [17], [24], [34], [36], [37], [38], [51]) whose source codes are not publicly available, we directly obtain their results from their published papers. For the other methods (i.e., [14], [19], [20], [21], [23], [50]), we directly use their official implementations for comparisons.…”

Section: Comparisons With Existing Methodsmentioning

confidence: 99%

3D Human Pose Machines with Self-supervised Learning

Wang

Jiang

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Driven by recent computer vision and robotic applications, recovering 3D human poses has become increasingly important and attracted growing interests. In fact, completing this task is quite challenging due to the diverse appearances, viewpoints, occlusions and inherently geometric ambiguities inside monocular images. Most of the existing methods focus on designing some elaborate priors /constraints to directly regress 3D human poses based on the corresponding 2D human pose-aware features or 2D pose predictions. However, due to the insufficient 3D pose data for training and the domain gap between 2D space and 3D space, these methods have limited scalabilities for all practical scenarios (e.g., outdoor scene). Attempt to address this issue, this paper proposes a simple yet effective self-supervised correction mechanism to learn all intrinsic structures of human poses from abundant images. Specifically, the proposed mechanism involves two dual learning tasks, i.e., the 2D-to-3D pose transformation and 3D-to-2D pose projection, to serve as a bridge between 3D and 2D human poses in a type of "free" self-supervision for accurate 3D human pose estimation. The 2D-to-3D pose implies to sequentially regress intermediate 3D poses by transforming the pose representation from the 2D domain to the 3D domain under the sequence-dependent temporal context, while the 3D-to-2D pose projection contributes to refining the intermediate 3D poses by maintaining geometric consistency between the 2D projections of 3D poses and the estimated 2D poses. Therefore, these two dual learning tasks enable our model to adaptively learn from 3D human pose data and external large-scale 2D human pose data. We further apply our self-supervised correction mechanism to develop a 3D human pose machine, which jointly integrates the 2D spatial relationship, temporal smoothness of predictions and 3D geometric knowledge. Extensive evaluations on the Human3.6M and HumanEva-I benchmarks demonstrate the superior performance and efficiency of our framework over all the compared competing methods. Please find the code of this project at: http://www.sysu-hcp.net/3d pose ssl/ Index Terms-human pose estimation, convolutional neural networks, spatio-temporal modeling, self-supervised learning, geometric deep learning. ! K. Wang is with the

show abstract

Monocular 3D Human Pose Estimation by Predicting Depth on Joints

Cited by 100 publications

References 25 publications

Hand Pose Estimation via Latent 2.5D Heatmap Regression

Hand Pose Estimation via Latent 2.5D Heatmap Regression

Can 3D Pose Be Learned from 2D Projections Alone?

3D Human Pose Machines with Self-supervised Learning

Contact Info

Product

Resources

About