GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping

Gundogdu, Erhan; Constantin, Victor; Seifoddini, Amrollah; Dang, Minh; Salzmann, Mathieu; Fua, Pascal

doi:10.1109/iccv.2019.00883

Cited by 143 publications

(119 citation statements)

References 35 publications

Supporting

Mentioning

119

Contrasting

Order By: Relevance

“…Data-driven clothing. A common strategy to learn ef-ficient data-driven models is to use off-line simulations [17,34,21,54,52,23] for generating data. These approaches often lack realism when compared to models trained using real data.…”

Section: Related Workmentioning

confidence: 99%

Multi-Garment Net: Learning to Dress 3D People From Images

Bhatnagar

Tiwari

Theobalt³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

349

134

View full text Add to dashboard Cite

We present Multi-Garment Network (MGN), a method to predict body shape and clothing, layered on top of the SMPL [40] model from a few frames (1-8) of a video. Several experiments demonstrate that this representation allows higher level of control when compared to single mesh or voxel representations of shape. Our model allows to predict garment geometry, relate it to the body shape, and transfer it to new body shapes and poses. To train MGN, we leverage a digital wardrobe containing 712 digital garments in correspondence, obtained with a novel method to register a set of clothing templates to a dataset of real 3D scans of people in different clothing and poses. Garments from the digital wardrobe, or predicted by MGN, can be used to dress any body shape in arbitrary poses. We will make publicly available the digital wardrobe, the MGN model, and code to dress SMPL with the garments at [1].

show abstract

Section: Related Workmentioning

confidence: 99%

Multi-Garment Net: Learning to Dress 3D People From Images

Bhatnagar

Tiwari

Theobalt³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

349

134

View full text Add to dashboard Cite

show abstract

“…The network determines if the features are correct, removing some of them if necessary. The last column indicates the ability of the learning approach managing the geometric operations that align datasets, such as computing the camera pose [27] or the transformation parameters [36].…”

Section: Review Of Learning-based Approaches For Registrationmentioning

confidence: 99%

“…Gundogdu et al [36] proposed a method for 3D garment fitting on bodies. To extract global features of the body model, they employed a PointNet [90] but with leaky ReLUs with a slope of 0.1.…”

Section: Transformation Levelmentioning

confidence: 99%

“…• Registration of global and local point clouds with a generic Deep Auto Encoder (DAE) architecture regardless of the input data and its source [15] • The Shape Deformation Network (SDN) is an autoencoder architecture able to deal with deformable shapes extracting global shape descriptors [29] • Capability for multi-spectral registration using Asymmetric Siamese Convolutional Networks [57] • Filtering inaccurate features correspondences based on geometrical and global properties to minimize their influence in the registration process [67,73] • Registration process speedup using a two-staged approach based on Convolutional Neural Networks (CNNs) to solve coarse-to-fine registration problems [63] transform • Using Spatial Transformer Networks (STNs) to deal with geometrical variability by means of learned invariance to transformations [45,47,95] • To align multimodal inputs with transformations generated and evaluated in an adversarial fashion [60] • Ability to preserve detailed geometric information by using a CNN to infer Free Form Deformation (FFD) parameters for 3D template-image matching [65] • Unsupervised registration of multiple point clouds in a global frame of reference [27] • Outperforming single channel CNNs with a multi-channel approach for real-time deformable registration [64] • Cloth fitting by modeling 3D body-garment interaction in real time surpassing Physics-Based Simulation (PBS) [36]…”

Section: Target Selectionmentioning

confidence: 99%

See 1 more Smart Citation

When Deep Learning Meets Data Alignment: A Review on Deep Registration Networks (DRNs)

et al. 2020

View full text Add to dashboard Cite

This paper reviews recent deep learning-based registration methods. Registration is the process that computes the transformation that aligns datasets, and the accuracy of the result depends on multiple factors. The most significant factors are the size of input data; the presence of noise, outliers and occlusions; the quality of the extracted features; real-time requirements; and the type of transformation, especially those defined by multiple parameters, such as non-rigid deformations. Deep Registration Networks (DRNs) are those architectures trying to solve the alignment task using a learning algorithm. In this review, we classify these methods according to a proposed framework based on the traditional registration pipeline. This pipeline consists of four steps: target selection, feature extraction, feature matching, and transform computation for the alignment. This new paradigm introduces a higher-level understanding of registration, which makes explicit the challenging problems of traditional approaches. The main contribution of this work is to provide a comprehensive starting point to address registration problems from a learning-based perspective and to understand the new range of possibilities.

show abstract

“…One stream of works try to build 3D human model and then render images [11,20,21,27], which can partially address the problem. However, it needs much efforts to collect the required information for a customer.…”

Section: Introductionmentioning

confidence: 99%

Down to the Last Detail

Wang

Sha

Zhang

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Virtual try-on has attracted lots of research attention due to its potential applications in e-commerce, virtual reality and fashion design. However, existing methods can hardly preserve the finegrained details (e.g., clothing texture, facial identity, hair style, skin tone) during generation, due to the non-rigid body deformation and multi-scale details. In this work, we propose a multi-stage framework to synthesize person images, where fine-grained details can be well preserved. To address the long-range translation and rich-details generation, we propose a Tree-Block (tree dilated fusion block) to replace standard ResNet-block where applicable. Notably, multi-scale feature maps can be smoothly fused for finegrained detail generation, by incorporating larger spatial context at multiple scales. With a delicate end-to-end training scheme, our whole framework can be jointly optimized for results with significantly better visual fidelity and richer details. Moreover, we also explore the potential application in video-based virtual try-on. By harnessing the well-trained image generator and an extra videolevel adaptor, a model photo can be well animated with a driving pose sequence. Extensive evaluations on standard datasets and user study demonstrate that our proposed framework achieves the stateof-the-art results, especially in preserving visual details in clothing texture and facial identity. Our implementation is publicly available via this URL.

show abstract

GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping

Cited by 143 publications

References 35 publications

Multi-Garment Net: Learning to Dress 3D People From Images

Multi-Garment Net: Learning to Dress 3D People From Images

When Deep Learning Meets Data Alignment: A Review on Deep Registration Networks (DRNs)

Down to the Last Detail

Contact Info

Product

Resources

About