DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras

Zheng, Yang; Shao, Ruizhi; Zhang, Yuxiang; Yu, Tao; Zheng, Zerong; Dai, Qionghai; Liu, Yebin

doi:10.1109/iccv48922.2021.00618

Cited by 81 publications

(20 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Global Pose Estimation. Most existing methods that estimate 3D poses in world coordinates rely on calibrated, synchronized, and static multi-view capture setups [5,13,15,29,38,76,77,113,114,116]. Huang et al [7] use uncalibrated cameras but still assume time synchronization and static camera setups.…”

Section: Related Workmentioning

confidence: 99%

GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras

Yuan¹,

Iqbal²,

Molchanov³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras

Yuan¹,

Iqbal²,

Molchanov³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, unposing clothed scans using the "undressed" model's skinning weights alters shape details. For the same RGB input, Zheng et al [52,53] condition the implicit function on a posed and voxelized SMPL mesh for robustness to pose variation, and reconstruct local details from the image pixels, similarly to PIFu [42]. However, these methods are sensitive to global pose, due to their 3D convolutional encoder.…”

Section: Related Workmentioning

confidence: 99%

ICON: Implicit Clothed humans Obtained from Normals

Xiu¹,

Yang²,

Tzionas³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Shape Representations of Humans. To capture detailed deformations of human bodies, most recent papers utilize implicit functions [37,38,7,47,56,57,19,58,39,68,45,67,78,20,77] or point clouds [34,36] due to their topological flexibility. These methods aim at learning geometry from 3D datasets, whereas we synthesize human images of novel poses only from 2D RGB training images.…”

Section: Related Workmentioning

confidence: 99%

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

Hu¹,

Chen²,

Zheng³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We propose a novel neural rendering pipeline, Hybrid Volumetric-Textural Rendering (HVTR), which synthesizes virtual human avatars from arbitrary poses efficiently and at high quality. First, we learn to encode articulated human motions on a dense UV manifold of the human body surface. To handle complicated motions (e.g., self-occlusions), we then leverage the encoded information on the UV manifold to construct a 3D volumetric representation based on a dynamic pose-conditioned neural radiance field. While this allows us to represent 3D geometry with changing topology, volumetric rendering is computationally heavy. Hence we employ only a rough volumetric representation using a pose-conditioned downsampled neural radiance field (PD-NeRF), which we can render efficiently at low resolutions. In addition, we learn 2D textural features that are fused with rendered volumetric features in image space. The key advantage of our approach is that we can then convert the fused features into a high resolution, high quality avatar by a fast GAN-based textural renderer. We demonstrate that hybrid rendering enables HVTR to handle complicated motions, render high quality avatars under usercontrolled poses/shapes and even loose clothing, and most importantly, be fast at inference time. Our experimental results also demonstrate state-of-the-art quantitative results. More results are available at our project page: https: // www. cs. umd. edu/ ~taohu/ hvtr/ .

show abstract

DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras

Cited by 81 publications

References 54 publications

GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras

GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras

ICON: Implicit Clothed humans Obtained from Normals

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

Contact Info

Product

Resources

About