3D Human Pose Estimation via Deep Learning from 2D Annotations

Brau, Ernesto; Jiang, Hao

doi:10.1109/3dv.2016.84

Cited by 48 publications

(30 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, the 3D pose can be predicted from a given set of 2D keypoints by simply predicting their depth [58]. Some works enforce priors about bone lengths and projection consistency with the 2D ground truth [2]. Video pose estimation.…”

Section: Related Workmentioning

confidence: 99%

“…For Mask R-CNN, we adopt a ResNet-101 backbone trained with the "stretched 1x" schedule [12]. 2 When finetuning the model on Human3.6M, we reinitialize the last layer of the keypoint network, as well as the deconv layers that regress the heatmaps to learn a new set of keypoints. We train on 4 GPUs with a step-wise decaying learning rate: 1e-3 for 60k iterations, then 1e-4 for 10k iterations, and 1e-5 for 10k iterations.…”

Section: Implementation Details For 2d Pose Estimationmentioning

confidence: 99%

See 1 more Smart Citation

3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training

Pavllo

Feichtenhofer

Grangier

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,015

909

View full text Add to dashboard Cite

In this work, we demonstrate that 3D poses in video can be effectively estimated with a fully convolutional model based on dilated temporal convolutions over 2D keypoints. We also introduce back-projection, a simple and effective semi-supervised training method that leverages unlabeled video data. We start with predicted 2D keypoints for unlabeled video, then estimate 3D poses and finally back-project to the input 2D keypoints. In the supervised setting, our fully-convolutional model outperforms the previous best result from the literature by 6 mm mean per-joint position error on Human3.6M, corresponding to an error reduction of 11%, and the model also shows significant improvements on HumanEva-I. Moreover, experiments with back-projection show that it comfortably outperforms previous state-of-the-art results in semisupervised settings where labeled data is scarce. Code and models are available at https://github.com/ facebookresearch/

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Implementation Details For 2d Pose Estimationmentioning

confidence: 99%

3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training

Pavllo

Feichtenhofer

Grangier

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,015

909

View full text Add to dashboard Cite

show abstract

“…Projection Layer ProjLayer: Recent work in 3D body pose estimation has integrated projection layers to leverage 2D-only annotated data for training 3D pose prediction [3]. Since our training dataset provides perfect 3D ground truth, we employ our projection layer merely as refinement module to link the 2D and 3D predictions.…”

Section: A2 Regnet Networkmentioning

confidence: 99%

GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB

Mueller

Bernard

Sotnychenko

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

489

497

View full text Add to dashboard Cite

Figure 1: We present an approach for real-time 3D hand tracking from monocular RGB-only input. Our method is compatible with unconstrained video input such as community videos from YouTube (left), and robust to occlusions (center-left). We show real-time 3D hand tracking results using an off-the-shelf RGB webcam in unconstrained setups (center-right, right). AbstractWe address the highly challenging problem of real-time 3D hand tracking based on a monocular RGB-only sequence. Our tracking method combines a convolutional neural network with a kinematic 3D hand model, such that it generalizes well to unseen data, is robust to occlusions and varying camera viewpoints, and leads to anatomically plausible as well as temporally smooth hand motions. For training our CNN we propose a novel approach for the synthetic generation of training data that is based on a geometrically consistent image-to-image translation network. To be more specific, we use a neural network that translates synthetic images to "real" images, such that the so-generated images follow the same statistical distribution as real-world hand images. For training this translation network we combine an adversarial loss and a cycle-consistency loss with a geometric consistency loss in order to preserve geometric properties (such as hand pose) during translation. We demonstrate that our hand tracking system outperforms the current state-of-the-art on challenging RGB-only footage.

show abstract

“…Existing approaches use bone length and depth ordering constraints [Mori and Malik 2006;Taylor 2000], sparsity assumptions [Wang et al 2014;Zhou et al 2015,a], joint limits [Akhter and Black 2015], inter-penetration constraints [Bogo et al 2016], temporal dependencies [Rhodin et al 2016b], and regression [Yasin et al 2016]. Treating 3D pose as a hidden variable in 2D estimation is an alternative [Brau and Jiang 2016]. However, the sparse set of 2D locations loses image evidence, e.g.…”

Section: Multi-viewmentioning

confidence: 99%

VNect

et al. 2017

View full text Add to dashboard Cite

Fig. 1. We recover the full global 3D skeleton pose in real-time from a single RGB camera, even wireless capture is possible by streaming from a smartphone (left). It enables applications such as controlling a game character, embodied VR, sport motion analysis and reconstruction of community video (right). Community videos (CC BY) courtesy of Real Madrid C.F. [2016] and RUSFENCING-TV [2017].We present the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera. Our method combines a new convolutional neural network (CNN) based pose regressor with kinematic skeleton fitting. Our novel fullyconvolutional pose formulation regresses 2D and 3D joint positions jointly in real time and does not require tightly cropped input frames. A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton. This makes our approach the first monocular RGB method usable in real-time applications such as 3D character control-thus far, the only monocular methods for such applications employed specialized RGB-D cameras. Our method's accuracy is quantitatively on par with the best offline 3D monocular RGB pose estimation methods. Our results are qualitatively comparable to, and sometimes better than, results from monocular RGB-D approaches, such as the Kinect. However, we show that our approach is more broadly applicable than RGB-D solutions, i.e., it works for outdoor scenes, community videos, and low quality commodity RGB cameras.

show abstract

3D Human Pose Estimation via Deep Learning from 2D Annotations

Cited by 48 publications

References 28 publications

3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training

3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training

GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB

VNect

Contact Info

Product

Resources

About