TexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video

Zhi, Tiancheng; Lassner, Christoph; Tung, Tony; Narasimhan, Srinivasa G.; Vo, Minh

doi:10.1007/978-3-030-58607-2_29

Cited by 32 publications

(21 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our proposed approach is trained solely on synthetic data and evaluated quantitatively and qualitatively on both, synthetic as well as real data. For synthetic data, we use the RenderPeople dataset [17], which has been used extensively [2,6,10,27,34,50,57,62,74] are static-the coverage of the pose space is still lacking. Hence, we augment the dataset by introducing additional pose variations: we perform non-rigid registration for all meshes, rig them for animation and use the Mixamo motion capture dataset [44] to animate them in an automatic fasion.…”

Section: Methodsmentioning

confidence: 99%

Free-Viewpoint RGB-D Human Performance Capture and Rendering

Nguyen-Ha¹,

Sarafianos²,

Lassner³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Methodsmentioning

confidence: 99%

Free-Viewpoint RGB-D Human Performance Capture and Rendering

Nguyen-Ha¹,

Sarafianos²,

Lassner³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other methods [Lähner et al 2018;Zhang et al 2020b] recover fine garment wrinkles for high-quality renderings or 3D modeling by augmenting a low-resolution normal map of a garment with high-frequency details using GANs. Zhi et al [2020] also reconstructs albedo textures and refines a coarse geometry obtained from RGB-D data. Our method factors cloth deformation into low-frequency large deformations represented by an embedded graph and high-frequency fine wrinkles modeled by a per-vertex displacements, which allows for synthesizing deformations for any kind of clothing, including also loose clothes.…”

Section: Related Workmentioning

confidence: 99%

Real-time deep dynamic characters

Habermann

Liu

et al. 2021

ACM Trans. Graph.

View full text Add to dashboard Cite

We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view imagery. In contrast to previous work, our controllable 3D character displays dynamics, e.g., the swing of the skirt, dependent on skeletal body motion in an efficient data-driven way, without requiring complex physics simulation. Our character model also features a learned dynamic texture model that accounts for photo-realistic motion-dependent appearance details, as well as view-dependent lighting effects. During training, we do not need to resort to difficult dynamic 3D capture of the human; instead we can train our model entirely from multi-view video in a weakly supervised manner. To this end, we propose a parametric and differentiable character representation which allows us to model coarse and fine dynamic deformations, e.g., garment wrinkles, as explicit spacetime coherent mesh geometry that is augmented with high-quality dynamic textures dependent on motion and view point. As input to the model, only an arbitrary 3D skeleton motion is required, making it directly compatible with the established 3D animation pipeline. We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing, including dynamics, and a neural generative dynamic texture model creates corresponding dynamic texture maps. We show that by merely providing new skeletal motions, our model creates motion-dependent surface deformations, physically plausible dynamic clothing deformations, as well as video-realistic surface textures at a much higher level of detail than previous state of the art approaches, and even in real-time.

show abstract

“…The final 3D pose was estimated by multi-view fusion and RGB-D optimization. Zhi et al [224] reconstructed detailed meshes with highresolution albedo texture from RGB-D video.…”

Section: D Hpe From Other Sourcesmentioning

confidence: 99%

Deep Learning-Based Human Pose Estimation: A Survey

Zheng¹,

Wu²,

Yang³

et al. 2020

Preprint

View full text Add to dashboard Cite

Human pose estimation aims to locate the human body parts and build human body representation (e.g., body skeleton) from input data such as images and videos. It has drawn increasing attention during the past decade and has been utilized in a wide range of applications including human-computer interaction, motion analysis, augmented reality, and virtual reality. Although the recently developed deep learning-based solutions have achieved high performance in human pose estimation, there still remain challenges due to insufficient training data, depth ambiguities, and occlusion. The goal of this survey paper is to provide a comprehensive review of recent deep learning-based solutions for both 2D and 3D pose estimation via a systematic analysis and comparison of these solutions based on their input data and inference procedures. More than 240 research papers since 2014 are covered in this survey. Furthermore, 2D and 3D human pose estimation datasets and evaluation metrics are included. Quantitative performance comparisons of the reviewed methods on popular datasets are summarized and discussed. Finally, the challenges involved, applications, and future research directions are concluded. We also provide a regularly updated project page: https://github.com/zczcwh/DL-HPE

show abstract

TexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video

Cited by 32 publications

References 50 publications

Free-Viewpoint RGB-D Human Performance Capture and Rendering

Free-Viewpoint RGB-D Human Performance Capture and Rendering

Real-time deep dynamic characters

Deep Learning-Based Human Pose Estimation: A Survey

Contact Info

Product

Resources

About