Can 3D Pose be Learned from 2D Projections Alone?

Drover, Dylan; Rohith, M.; Chen, Ching-Hang; Agrawal, Amit; Tyagi, Ambrish; Huynh, Cong Phuoc

doi:10.48550/arxiv.1808.07182

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even simple architectures have been shown to solve this task with a low error rate [26]. Recent advances are due to combining the 2D and 3D tasks into a joint estimation [41,42] and using weakly [54,48,50,9,33] or selfsupervised losses [49,38] or mixing 2D and 3D data for training [46].…”

Section: Related Workmentioning

confidence: 99%

xR-EgoPose: Egocentric 3D Human Pose From an HMD Camera

Tomè

Peluse

Agapito

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

104

View full text Add to dashboard Cite

Figure 1: Left: Our xR-EgoPose Dataset setup: (a) external camera viewpoint showing a synthetic character wearing the headset; (b) example of photorealistic image rendered from the egocentric camera perspective; (c) 2D and (d) 3D poses estimated with our algorithm. Right: results on real images; (e) real image acquired with our HMD-mounted camera with predicted 2D heatmaps; (f) estimated 3D pose, showing good generalization to real images. AbstractWe present a new solution to egocentric 3D body pose estimation from monocular images captured from a downward looking fish-eye camera installed on the rim of a head mounted virtual reality device. This unusual viewpoint, just 2 cm. away from the user's face, leads to images with unique visual appearance, characterized by severe self-occlusions and strong perspective distortions that result in a drastic difference in resolution between lower and upper body. Our contribution is two-fold. Firstly, we propose a new encoderdecoder architecture with a novel dual branch decoder designed specifically to account for the varying uncertainty in the 2D joint locations. Our quantitative evaluation, both on synthetic and real-world datasets, shows that our strategy leads to substantial improvements in accuracy over state of the art egocentric pose estimation approaches. Our second contribution is a new large-scale photorealistic synthetic dataset -xR-EgoPose -offering 383K frames of high quality renderings of people with a diversity of skin tones, body shapes, clothing, in a variety of backgrounds and lighting conditions, performing a range of actions. Our experiments show that the high variability in our new synthetic training corpus leads to good generalization to real world footage and to state of the art results on real world datasets with ground truth. Moreover, an evaluation on the Human3.6M benchmark shows that the performance of our method is on par with top performing approaches on the more classic problem of 3D human pose from a third person viewpoint.

show abstract