Head3D: Complete 3D Head Generation via Tri-plane Feature Distillation

Abstract

Head generation with diverse identities is an important task in computer vision and computer graphics, widely used in multimedia applications. However, current full head generation methods require a large number of 3D scans or multi-view images to train the model, resulting in expensive data acquisition cost. To address this issue, we propose Head3D, a method to generate full 3D heads with limited multi-view images. Specifically, our approach first extracts facial priors represented by tri-planes learned in EG3D, a 3D-aware generative model, and then proposes feature distillation to deliver the 3D frontal faces into complete heads without compromising head integrity. To mitigate the domain gap between the face and head models, we present dual-discriminators to guide the frontal and back head generation, respectively. Our model achieves cost-efficient and diverse complete head generation with photo-realistic renderings and high-quality geometry representations. Extensive experiments demonstrate the effectiveness of our proposed Head3D, both qualitatively and quantitatively.

Recent advances in implicit representation-based 3D shape generation

“…General objects [103][104][105][106][107][108][109] [ 110-114] Human bodies [115][116][117] -Human faces [13,[118][119][120][121][122][123][124][125][126][127][128] [ 129] Scenes [130] editing of generation results by utilizing appearance codes and semantic masks. DATID-3D [120] aims to transfer EG3D to another domain, e.g., animation.…”

“…PanoHead [126] generates 3D human heads from 360°full head images, using a self-adaptive image alignment and a tri-grid volume to solve the "mirrored face" artifact of EG3D. Head3D [127] utilizes a teacher-student distillation technique and a dual-discriminator structure to solve the front-back gap for full-head generation present in EG3D-based methods. GINA-3D [108] decouples representation learning and generation, and uses VAE to map input images to latent feature represented by triplanes using quantization, cross-attention, and neural rendering.…”