We present a learning‐based approach for virtual try‐on applications based on a fully convolutional graph neural network. In contrast to existing data‐driven models, which are trained for a specific garment or mesh topology, our fully convolutional model can cope with a large family of garments, represented as parametric predefined 2D panels with arbitrary mesh topology, including long dresses, shirts, and tight tops. Under the hood, our novel geometric deep learning approach learns to drape 3D garments by decoupling the three different sources of deformations that condition the fit of clothing: garment type, target body shape, and material. Specifically, we first learn a regressor that predicts the 3D drape of the input parametric garment when worn by a mean body shape. Then, after a mesh topology optimization step where we generate a sufficient level of detail for the input garment type, we further deform the mesh to reproduce deformations caused by the target body shape. Finally, we predict fine‐scale details such as wrinkles that depend mostly on the garment material. We qualitatively and quantitatively demonstrate that our fully convolutional approach outperforms existing methods in terms of generalization capabilities and memory requirements, and therefore it opens the door to more general learning‐based models for virtual try‐on applications.

Gr o u n d T r u t h S y n t h e s i z e d S y n t h e s i z e d P r e d i c t i o n I n p u t I ma g e s I n p u t P h o t o sFigure 1: From just two input images (left) our method is capable of estimating the BRDF parameters (right, synthesized from novel viewpoints) of complex materials. AbstractThe estimation of the optical properties of a material from RGB-images is an important but extremely ill-posed problem in Computer Graphics. While recent works have successfully approached this problem even from just a single photograph, significant simplifications of the material model are assumed, limiting the usability of such methods. The detection of complex material properties such as anisotropy or Fresnel effect remains an unsolved challenge. We propose a novel method that predicts the model parameters of an artist-friendly, physically-based BRDF, from only two low-resolution shots of the material. Thanks to a novel combination of deep neural networks in a nested architecture, we are able to handle the ambiguities given by the nonorthogonality and non-convexity of the parameter space. To train the network, we generate a novel dataset of physicallybased synthetic images. We prove that our model can recover new properties like anisotropy, index of refraction and a second reflectance color, for materials that have tinted specular reflections or whose albedo changes at glancing angles.

Figure 1: Our method predicts the 3D draping for an arbitrary body shape and garment parameters at interactive rates. From left to right, a variety of body shapes obtained from a parametric avatar model, different 2D panel configurations of our paremeterized garment types, and corresponding dressed 3D bodies generated with our novel fully convolutional approach.

No abstract