Driving-signal aware full-body avatars

Structured Local Radiance Fields for Human Avatar Modeling

“…Although we can obtain these variables for training frames through naive optimization with image evidence, it remains unclear how to compute them for unseen poses. Alternatively, one can train a network that directly regresses these variables from body poses, but this will result into the aforementioned under-fitting issues due to information deficiency [6]. In order to achieve a balance between data fitting and generalization, we draw inspiration from [6] and learn the node-related variables in a conditional generative latent space.…”

“…Alternatively, one can train a network that directly regresses these variables from body poses, but this will result into the aforementioned under-fitting issues due to information deficiency [6]. In order to achieve a balance between data fitting and generalization, we draw inspiration from [6] and learn the node-related variables in a conditional generative latent space. Specifically, we introduce a tiny conditional variational auto-encoder (cVAE) [69] for each local radiance field.…”

“…To facilitate geometric learning, several statistical parametric templates are developed for face [31], hands [48,61] and minimally clothed body [28,41,51,54]. To acquire animatable characters wearing casual clothes, traditional pipelines mostly reconstruct a subject-specific mesh template in advance, and then generate its motions using physics simulation [20,71], deformation space modeling [28], or deep learning [6,22,79]. The reliance on pre-scanning efforts can be eliminated via deforming a general body template, and several works proposed to directly learn this deformation from geometric data [42][43][44]57] or RGB videos [2][3][4][5].…”

“…Traditional methods for high-quality human avatar reconstruction are often costly and tedious, due to the difficulties in modeling the complex dynamics of clothes. Besides, they typically presume the availability of a subjectspecific template [22] and its accurate registration to the input frames [6,79], which are difficult to acquire in practice.…”

Garment Avatars: Realistic Cloth Driving using Pattern Registration

“…Truthful representation of humans is essential for virtual telepresence applications, the future of communication and avatars in the metaverse. Previous efforts have focused primarily on human heads [46] and bodies [3,59]. Due to the highly complicated motion of garments, it still remains elusive to teleport the clothing faithfully on a body avatar.…”