Towards Multi-Pose Guided Virtual Try-On Network

Abstract

Virtual try-on system under arbitrary human poses has huge application potential, yet raises quite a lot of challenges, e.g. self-occlusions, heavy misalignment among diverse poses, and diverse clothes textures. Existing methods aim at fitting new clothes into a person can only transfer clothes on the fixed human pose, but still show unsatisfactory performances which often fail to preserve the identity, lose the texture details, and decrease the diversity of poses. In this paper, we make the first attempt towards multi-pose guided virtual try-on system, which enables transfer clothes on a person image under diverse poses. Given an input person image, a desired clothes image, and a desired pose, the proposed Multi-pose Guided Virtual Try-on Network (MG-VTON) can generate a new person image after fitting the desired clothes into the input image and manipulating human poses. Our MG-VTON is constructed in three stages: 1) a desired human parsing map of the target image is synthesized to match both the desired pose and the desired clothes shape; 2) a deep Warping Generative Adversarial Network (Warp-GAN) warps the desired clothes appearance into the synthesized human parsing map and alleviates the misalignment problem between the input human pose and desired human pose; 3) a refinement render utilizing multi-pose composition masks recovers the texture details of clothes and removes some artifacts. Extensive experiments on well-known datasets and our newly collected largest virtual try-on benchmark demonstrate that our MG-VTON significantly outperforms all state-of-the-art methods both qualitatively and quantitatively with promising multipose virtual try-on performances.

UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person Features

“…The final resolution of the C2C market images is 128 × 96 due to adding white margins to the images. For testing virtual try-on, we also couple our collected clothing only images with 10,322 front-view person images from the Multi-Pose Virtual try-on datasets [ 10 ]. In Figure 2 , we can see that the C2C market dataset images are not aligned (e.g., hanged, folded).…”

“…We also conduct experiments to compare conventional supervised virtual try-on methods and our UVIRT method on the Multi-Pose Virtual try-on (MPV) datasets collected by Dong et al [ 10 ]. It contains 35,687 person and 13,524 clothing images at 256 × 192 resolution.…”

“…In the experiments section, we first qualitatively and quantitatively compare UVIRT with the conventional methods by evaluating on the MPV dataset [ 10 ]. We then qualitatively and quantitatively confirm that UVIRT can be applied to an in-the-wild C2C market image dataset collected by us.…”

Down to the Last Detail

“…However, they can only present on a certain viewpoint and can not flexibly apply to arbitrary poses. To apply arbitrary poses and clothes to a certain person, MG-VTON [7] proposed to warp warps the desired clothes appearance into the synthesized human parsing map and alleviated the misalignment problem between the input human pose and desired human pose. There also exists method [42] which used a attentive bidirectional GAN to better refine the quality of person image in two stage.…”

“…Such huge labor costs and expensive devices also limit the applications in practice. Another 2D-based stream of works [7,42] maintain the cloth details through feature warping and bi-directional framework, but neglect the weakness of processing long-range information for standard convolutional operation.…”