Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild

Sengupta, Akash; Budvytis, Ignas; Cipolla, Roberto

doi:10.1109/iccv48922.2021.01103

Cited by 47 publications

(46 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The edge-image and joint heatmaps of I I I n ∈ R H×W ×3 are stacked to form a proxy representation X X X n ∈ R H×W ×(J+1) (for J joints). We use this proxy representation as our input, instead of the RGB image, to decrease the domain gap between synthetic training images [37,38,39] and real test images. Body measurements and pose distribution prediction.…”

Section: Methodsmentioning

confidence: 99%

“…Learning-based methods yield impressive 3D pose estimates in-the-wild, but shape predictions are often inaccurate, due to the lack of shape diversity in training datasets. Some recent works improve shape estimates using synthetic training data [37,38,39,40], which we adopt in our method.…”

Section: Related Workmentioning

confidence: 99%

“…The bottom half presents metrics from competing approaches. Note that our 10 β s Net is equivalent to [39], except we use improved edge-based training inputs [38]. All numbers in mm.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…However, body shape estimates tend to be inaccurate or inconsistent for subjects in-the-wild. Recently, [38,39] attempt to predict accurate and consistent body shapes from multiple images of a subject, without assuming a fixed body pose or background and lighting conditions. This involves (i) predicting independent Gaussian distributions (i.e.…”

Section: Introductionmentioning

confidence: 99%

“…First, we compute proxy representations using an off-the-shelf 2D keypoint detector [14,46] and Canny edge detection [6]. This decreases the domain gap between synthetic training data and real test data [38]. Second, a deep neural network predicts means and variances of Gaussian distributions over SMPL pose parameters and body measurements, conditioned on the input proxy representations.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Probabilistic Estimation of 3D Human Shape and Pose with a Semantic Local Parametric Model

Sengupta¹,

Budvytis²,

Cipolla³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper addresses the problem of 3D human body shape and pose estimation from RGB images. Some recent approaches to this task predict probability distributions over human body model parameters conditioned on the input images. This is motivated by the ill-posed nature of the problem wherein multiple 3D reconstructions may match the image evidence, particularly when some parts of the body are locally occluded. However, body shape parameters in widely-used body models (e.g. SMPL) control global deformations over the whole body surface. Distributions over these global shape parameters are unable to meaningfully capture uncertainty in shape estimates associated with locallyoccluded body parts. In contrast, we present a method that (i) predicts distributions over local body shape in the form of semantic body measurements and (ii) uses a linear mapping to transform a local distribution over body measurements to a global distribution over SMPL shape parameters. We show that our method outperforms the current stateof-the-art in terms of identity-dependent body shape estimation accuracy on the SSP-3D dataset, and a private dataset of tape-measured humans, by probabilistically-combining local body measurement distributions predicted from multiple images of a subject.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…The bottom half presents metrics from competing approaches. Note that our 10 β s Net is equivalent to [39], except we use improved edge-based training inputs [38]. All numbers in mm.…”

Section: Implementation Detailsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Probabilistic Estimation of 3D Human Shape and Pose with a Semantic Local Parametric Model

Sengupta¹,

Budvytis²,

Cipolla³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Modification and refinement of three‐dimensional reconstruction to estimate body volume from a simulated single‐camera image

Chiu

Dunn

Heller

et al. 2022

Obesity Science & Practice

View full text Add to dashboard Cite

Objective: Body volumes (BV) are used for calculating body composition to perform obesity assessments. Conventional BV estimation techniques, such as underwater weighing, can be difficult to apply. Advanced machine learning techniques enable multiple obesity-related body measurements to be obtained using a single-camera image; however, the accuracy of BV calculated using these techniques is unknown. This study aims to adapt and evaluate a machine learning technique, synthetic training for real accurate pose and shape (STRAPS), to estimate BV. Methods:The machine learning technique, STRAPS, was applied to generate threedimensional (3D) models from simulated two-dimensional (2D) images; these 3D models were then scaled with body stature and BV were estimated using regression models corrected for body mass. A commercial 3D scan dataset with a wide range of participants (n = 4318) was used to compare reference and estimated BV data. Results:The developed methods estimated BV with small relative standard errors of estimation (<7%) although performance varied when applied to different groups.The BV estimated for people with body mass index (BMI) < 30 kg/m 2 (1.9% for males and 1.8% for females) were more accurate than for people with BMI ≥ 30 kg/m 2 (6.9% for males and 2.4% for females). Conclusions:The developed method can be used for females and males with BMI < 30 kg/m 2 in BV estimation and could be used for obesity assessments at home or clinic settings.

show abstract

Self-supervised Human Mesh Recovery with Cross-Representation Alignment

Gong

Zheng²,

Planche³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild

Cited by 47 publications

References 39 publications

Probabilistic Estimation of 3D Human Shape and Pose with a Semantic Local Parametric Model

Probabilistic Estimation of 3D Human Shape and Pose with a Semantic Local Parametric Model

Modification and refinement of three‐dimensional reconstruction to estimate body volume from a simulated single‐camera image

Self-supervised Human Mesh Recovery with Cross-Representation Alignment

Contact Info

Product

Resources

About