Playing for 3D Human Recovery

Cai, Zhongang; Zhang, Mingyuan; Ren, Jiawei; Chen, Wei; Ren, Daxuan; Li, Jiatong; Lin, Zhengyu; Zhao, Haiyu; Yi, Shuai; Yang, Lei; Loy, Chen Change; Liu, Ziwei

doi:10.48550/arxiv.2110.07588

Cited by 4 publications

(4 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dataset contains ground truth depth maps, optical flow, surface normals, human part segmentations, and 2D/3D joint locations. GTA-Human [209] is a large-scale 3D human dataset with a diverse set of subjects, actions, and scenarios generated with the GTA-V game engine. There are 20K video sequences with SMPL annotations in this dataset.…”

Section: Rendered Datasetsmentioning

confidence: 99%

Recovering 3D Human Mesh from Monocular Images: A Survey

Tian¹,

Zhang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Estimating human pose and shape from monocular images is a long-standing problem in computer vision. Since the release of statistical body models, 3D human mesh recovery has been drawing broader attention. With the same goal of obtaining well-aligned and physically plausible mesh results, two paradigms have been developed to overcome challenges in the 2D-to-3D lifting process: i) an optimization-based paradigm, where different data terms and regularization terms are exploited as optimization objectives; and ii) a regression-based paradigm, where deep learning techniques are embraced to solve the problem in an end-to-end fashion. Meanwhile, continuous efforts are devoted to improving the quality of 3D mesh labels for a wide range of datasets. Though remarkable progress has been achieved in the past decade, the task is still challenging due to flexible body motions, diverse appearances, complex environments, and insufficient in-the-wild annotations. To the best of our knowledge, this is the first survey to focus on the task of monocular 3D human mesh recovery. We start with the introduction of body models and then elaborate recovery frameworks and training objectives by providing in-depth analyses of their strengths and weaknesses. We also summarize datasets, evaluation metrics, and benchmark results. Open issues and future directions are discussed in the end, hoping to motivate researchers and facilitate their research in this area. A regularly updated project page can be found at https://github.com/tinatiansjz/hmr-survey.

show abstract

Section: Rendered Datasetsmentioning

confidence: 99%

Recovering 3D Human Mesh from Monocular Images: A Survey

Tian¹,

Zhang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, 3D rendering tools employed in video games have become a valuable source for collecting synthetic data, aiming to improve performances across different human analysis tasks. Among others, Zhu et al [67] and Cai et al [68] extracted training data from NBA2K2019 and GTA-V, in order to achieve state-of-the-art performances in 3D human body reconstruction. Other studies exploiting 3D rendering tools for generating synthetic data spanned applications in re-identification of individuals [78], face recognition [79], [52], and gait recognition [80].…”

Section: Methodsmentioning

confidence: 99%

Synthetic Data in Human Analysis: A Survey

Joshi,

Grimmer,

Rathgeb

et al. 2024

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Deep neural networks have become prevalent in human analysis, boosting the performance of applications, such as biometric recognition, action recognition, as well as person re-identification. However, the performance of such networks scales with the available training data. In human analysis, the demand for large-scale datasets poses a severe challenge, as data collection is tedious, time-expensive, costly and must comply with data protection laws. Current research investigates the generation of synthetic data as an efficient and privacy-ensuring alternative to collecting real data in the field. This survey introduces the basic definitions and methodologies, essential when generating and employing synthetic data for human analysis. We summarise current state-of-the-art methods and the main benefits of using synthetic data. We also provide an overview of publicly available synthetic datasets and generation models. Finally, we discuss limitations, as well as open research problems in this field. This survey is intended for researchers and practitioners in the field of human analysis.

show abstract

“…Synthetic dataset such as AGORA [77] renders high-quality human scans in virtual environments and fits SMPL to the original mesh. Video games have also become an alternative source of data [9,10]. In addition to SMPL parameters that do not model clothes or texture, HuMMan also provides textured meshes of clothed subjects.…”

Section: Related Workmentioning

confidence: 99%

HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling

Cai¹,

Ren²,

Zeng³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Point Cloud c) Keypoints d) SMPL e) Mesh f) Texture a) Color ImageFigure 1. HuMMan features multiple modalities of data format and annotations. We demonstrate a) color image, b) point cloud, c) keypoints, d) SMPL parameters and e) mesh geometry with f) texture. Each sequence is also annotated with an action label from 500 actions. Each subject has two additional high-resolution scans of naturally and minimally clothed body.

show abstract

Playing for 3D Human Recovery

Cited by 4 publications

References 62 publications

Recovering 3D Human Mesh from Monocular Images: A Survey

Recovering 3D Human Mesh from Monocular Images: A Survey

Synthetic Data in Human Analysis: A Survey

HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling

Contact Info

Product

Resources

About