Learning to Dress 3D People in Generative Clothing

Abstract

Three-dimensional human body models are widely used in the analysis of human pose and motion. Existing models, however, are learned from minimally-clothed 3D scans and thus do not generalize to the complexity of dressed people in common images and videos. Additionally, current models lack the expressive power needed to represent the complex non-linear geometry of pose-dependent clothing shapes. To address this, we learn a generative 3D mesh model of clothed people from 3D scans with varying pose and clothing. Specifically, we train a conditional Mesh-VAE-GAN to learn the clothing deformation from the SMPL body model, making clothing an additional term in SMPL. Our model is conditioned on both pose and clothing type, giving the ability to draw samples of clothing to dress different body shapes in a variety of styles and poses. To preserve wrinkle detail, our Mesh-VAE-GAN extends patchwise discriminators to 3D meshes. Our model, named CAPE, represents global shape and fine local structure, effectively extending the SMPL body model to clothing. To our knowl-edge, this is the first generative model that directly dresses 3D human body meshes and generalizes to different poses. The model, code and data are available for research purposes at https://cape.is.tue.mpg.de.

PLACE: Proximity Learning of Articulation and Contact in 3D Environments

“…Binary user study. To perform a direct comparison, we follow [21,25], and show results of two methods to the turkers at the same time. The turkers are asked to pick the one that they think is more perceptually natural.…”

Image-Guided Human Reconstruction via Multi-Scale Graph Transformation Networks

“…All the above methods do not provide topology-consistent dynamic human meshes with detailed surface geometry, which are very useful for training and evaluating graph-based deep neural networks. The CAPE [48] dataset is a dynamic 3D clothed human model dataset with consistent SMPL mesh topology (6890 vertices), but the geometry details are limited due to using a small number of vertices. Moreover, the color images are re-rendered, not real captured.…”

Reconstructing Human Body Mesh from Point Clouds by Adversarial GP Network

“…In 3D vision, HMR [34] applies adversarial learning to estimate 3D human body shape and pose from 2D images. CAPE [35] uses it to learn a model of people in clothing. Fernández Abrevaya et al [36] and Shamai et al [37] use adversarial training to model faces in 3D.…”