Neural Capture of Animatable 3D Human from Monocular Video

Te, Gusi; Li, Xiu; Li, Xiao; Wang, Jinglu; Hu, Wei; Lu, Yan

doi:10.1007/978-3-031-20068-7_16

Cited by 16 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mildenhall et al [36,68] pioneered NeRFs for representing static scenes with a color and density field without requiring any 3D ground truth supervision. Recently this approach has been extended to reconstruct clothed humans as well [43,58,12,67,24,66,30,65,23,19,62]. These approaches use SMPL as a prior to unpose the human body across multiple frames by transforming the rays from observation space to canonical space which is then rendered using a NeRF.…”

Section: Human Reconstruction Via Optimizationmentioning

confidence: 99%

Mesh Strikes Back: Fast and Efficient Human Reconstruction from RGB videos

Jena¹,

Chaudhari²,

Gee³

et al. 2023

Preprint

View full text Add to dashboard Cite

Human reconstruction and synthesis from monocular RGB videos is a challenging problem due to clothing, occlusion, texture discontinuities and sharpness, and framespecific pose changes. Many methods employ deferred rendering, NeRFs and implicit methods to represent clothed humans, on the premise that mesh-based representations cannot capture complex clothing and textures from RGB, silhouettes, and keypoints alone. We provide a counter viewpoint to this fundamental premise by optimizing a SMPL+D mesh and an efficient, multi-resolution texture representation using only RGB images, binary silhouettes and sparse 2D keypoints. Experimental results demonstrate that our approach is more capable of capturing geometric details compared to visual hull, mesh-based methods. We show competitive novel view synthesis and improvements in novel pose synthesis compared to NeRF-based methods, which introduce noticeable, unwanted artifacts. By restricting the solution space to the SMPL+D model combined with differentiable rendering, we obtain dramatic speedups in compute, training times (up to 24x) and inference times (up to 192x). Our method therefore can be used as is or as a fast initialization to NeRF-based methods.

show abstract

Section: Human Reconstruction Via Optimizationmentioning

confidence: 99%

Mesh Strikes Back: Fast and Efficient Human Reconstruction from RGB videos

Jena¹,

Chaudhari²,

Gee³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Avatar reconstruction of the human body. Prominent techniques for human body reconstruction necessitate expensive data acquisition, e.g., multi-view RGB video [10,21,28,29,39,40,53,54,63,73,81,86,88,89], monocular RGB video [8,19,20,31,34,35,70,76,83,87], textured scan video [22,64], or image set [77]. Also, many of them utilize NeRF or its variations as the 3D representation and craft a motion field to connect the gap between body articulation and the canonical NeRF space.…”

Section: Related Workmentioning

confidence: 99%