Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

Fan, Taosha; Alwala, Kalyan Vasudev; Xiang, Donglai; Xu, Weipeng; Murphey, Todd D.; Mukadam, Mustafa

doi:10.1109/iccv48922.2021.01126

Cited by 22 publications

(11 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Output Type a) Template paramters: [16], [17], [18], [19], [20], [21], [25], [37], [94], [97], [98], [99], [101], [102], [103], [104], [105], [106], [111], [112], [125], [129], [135], [140] b) 3D vertex coordinates: GraphCMR [117], Pose2Mesh [132], I2L-MeshNet [118], PC-HMR [121], METRO [120], Graphormer [119] c) Voxels: BodyNet [115], DeepHuman [116] d) UV position maps: DenseBody [122], DecoMR [123], Zhang et al [124] e) Probabilistic outputs: Biggs et al [125], Sengupta et al [127], [128], ProHMR [126] Intermediate/ Proxy Representation a) Silhouettes: Pavlakos et al [17], STRAPS [103], Skeleton2Mesh [98] b) Segmentations: NBF [18], Rueegg et al [129], STRAPS [103], Zanfir et al [105], HUND…”

Section: Framebased Single Personmentioning

confidence: 99%

See 1 more Smart Citation

Recovering 3D Human Mesh from Monocular Images: A Survey

Tian¹,

Zhang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Estimating human pose and shape from monocular images is a long-standing problem in computer vision. Since the release of statistical body models, 3D human mesh recovery has been drawing broader attention. With the same goal of obtaining well-aligned and physically plausible mesh results, two paradigms have been developed to overcome challenges in the 2D-to-3D lifting process: i) an optimization-based paradigm, where different data terms and regularization terms are exploited as optimization objectives; and ii) a regression-based paradigm, where deep learning techniques are embraced to solve the problem in an end-to-end fashion. Meanwhile, continuous efforts are devoted to improving the quality of 3D mesh labels for a wide range of datasets. Though remarkable progress has been achieved in the past decade, the task is still challenging due to flexible body motions, diverse appearances, complex environments, and insufficient in-the-wild annotations. To the best of our knowledge, this is the first survey to focus on the task of monocular 3D human mesh recovery. We start with the introduction of body models and then elaborate recovery frameworks and training objectives by providing in-depth analyses of their strengths and weaknesses. We also summarize datasets, evaluation metrics, and benchmark results. Open issues and future directions are discussed in the end, hoping to motivate researchers and facilitate their research in this area. A regularly updated project page can be found at https://github.com/tinatiansjz/hmr-survey.

show abstract

Section: Framebased Single Personmentioning

confidence: 99%

“…Inspired by this, Biggs et al [125] also adopt the Real-NVP architecture. Fan et al [140] design a normalizing flow using fully-connected layers. The GHUM/GHUML models [81] rely on normalizing flows to represent skeleton kinematics.…”

Section: Pose Prior and Shape Priormentioning

confidence: 99%

Recovering 3D Human Mesh from Monocular Images: A Survey

Tian¹,

Zhang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, adapting the optimizer to run in real-time is a nontrivial operation, due to the cubic complexity of the popular Levenberg-Marquardt algorithm [26,38,44]. The most common and practical way to speedup the optimization is to utilize the sparsity of the problem or make certain assumptions to simplify it [17]. Learned optimizers promise to overcome these issues, by learning the parametric model priors directly from the data and to take more aggressive steps, thus converging in fewer iterations.…”

Section: Related Workmentioning

confidence: 99%

Learning to Fit Morphable Models

Choutas¹,

Bogo²,

Shen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Figure 1. Accurate head and hand poses are becoming readily available on AR and VR headsets. One current challenge to increase immersion in virtual worlds consists in fitting 3D body models to these sparse measurements (top left) which is a problem often tackled using regression followed by iterative mathematical optimization (top right). While these approaches have demonstrated their effectiveness on many problems, they require significant amounts of manual labor to work at speed and quality. Using machine learning for continuous optimization offers the opportunity to overcome this hurdle by learning the priors directly from the data and by leveraging optimized backend implementations (bottom left). In this way, we can construct effective models, without the biases introduced by the manually defined energy terms. The resulting models are fast, able to tightly fit the data (bottom right, ground-truth mesh in blue) and can easily be trained for different problem setups. Note that all results are estimated independently per-frame.

show abstract

“…Recent years have witnessed significant development of marker-less motion capture, which promotes a wide variety of applications ranging from character animation to human-computer interaction, personal well-being, and human behavior understanding. Extensive existing works can kinematically capture accurate human pose from monocular videos and images via network regression [23,26,27,72,73] or optimization [6,42,45,55]. However, they are often hard to leverage in real-world systems due to a series of artifacts that are not satisfied biomechanical and physical plausibility (e.g., jitter and floor penetration).…”

Section: Introductionmentioning

confidence: 99%

Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture

Huang¹,

Peng²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Due to the visual ambiguity, purely kinematic formulations on monocular human motion capture are often physically incorrect, biomechanically implausible, and can not reconstruct accurate interactions. In this work, we focus on exploiting the high-precision and non-differentiable physics simulator to incorporate dynamical constraints in motion capture. Our key-idea is to use real physical supervisions to train a target pose distribution prior for sampling-based motion control to capture physically plausible human motion. To obtain accurate reference motion with terrain interactions for the sampling, we first introduce an interaction constraint based on SDF (Signed Distance Field) to enforce appropriate ground contact modeling. We then design a novel two-branch decoder to avoid stochastic error from pseudo ground-truth and train a distribution prior with the non-differentiable physics simulator. Finally, we regress the sampling distribution from the current state of the physical character with the trained prior and sample satisfied target poses to track the estimated reference motion. Qualitative and quantitative results show that we can obtain physically plausible human motion with complex terrain interactions, human shape variations, and diverse behaviors. More information can be found at https:// www.yangangwang.com/papers/HBZ-NM-2022-03.html

show abstract

Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

Cited by 22 publications

References 36 publications

Recovering 3D Human Mesh from Monocular Images: A Survey

Recovering 3D Human Mesh from Monocular Images: A Survey

Learning to Fit Morphable Models

Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture

Contact Info

Product

Resources

About