Neural Head Avatars from Monocular RGB Videos

MEGANE: Morphable Eyeglass and Avatar Network

“…Bi et al [3] enable high-fidelity relighting of photorealistic avatars in real-time. While the aforementioned approaches require multi-view capture systems, recent works show that modeling of photorealistic avatars from monocular video inputs is also possible [1,15,19]. Cao et al [6] recently extend these personspecific neural rendering approaches to a multi-identity model, and demonstrates the personalized adaptation of the learned universal morphable model from a mobile phone scan.…”

“…Recently, neural rendering approaches [60] achieve photorealistic rendering of human heads [15,19,37,38,51] and general objects [43,47,64,74] in a 3D consistent manner. These approaches are further extended to generative modeling for faces [6] and glasses [42,68], such that a single morphable model can span the shape and appearance variation of each object category.…”

Efficient Meshy Neural Fields for Animatable Human Avatars

“…Thus, we can learn both geometry and textures that are compatible with existing graphics hardware. However, the geometry optimization process is nonconvex and highly unstable [23], so it is hard to give finegrained geometry details. Besides, the topology of the mesh is fixed leading to limited shape modeling.…”

KeypointNeRF: Generalizing Image-Based Volumetric Avatars Using Relative Spatial Encoding of Keypoints

“…Learning Face and Body Reconstruction: Generalizable parametric mesh [4,32,66] and implicit [2,36,38,61,65] body models can provide additional constraints for learning details from sparse views. Recent approaches have incorporated priors specific to human faces [8,9,15,17,48,59,70] and human bodies [42,43,62,64,65,69,71,72] to reduce the dependence on multi-view captures. Approaches such as H3DNet [47] and SIDER [12] use signed-distance functions (SDFs) for learning geometry priors from large datasets and perform test-time fine-tuning on the test subject.…”