Human Appearance Transfer

Zanfir, Mihai; Popa, Alin-Ionut; Zanfir, Andrei; Sminchisescu, Cristian

doi:10.1109/cvpr.2018.00565

Cited by 107 publications

(85 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This allows to map the body pixels to a meaningful UV-coordinate space, where surface interpolation and inpainting can happen before warping back to the image space. Zanfir [44] on the other hand, proposed to leverage 3D human model to explicitly capture the body deformations. Specifically, they fit a 3D human model [21] to both source and target images using the method in [43], where a human body is represented by 6890 surface vertices.…”

Section: Related Workmentioning

confidence: 99%

“…Neverova et al [26] showed that the surface-based pose representation via DensePose [7] serves as a better alternative. Zanfir et al [44] turned to fit a 3D model to both input and target images, and then perform appearance transfer between the corresponding vertices. The resulting appearance flow with 3D geometry supervision is more ideal, but the 3D model fitting would incur too much burden at inference time.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dense Intrinsic Appearance Flow for Human Pose Transfer

Huang

Loy

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

216

144

View full text Add to dashboard Cite

We present a novel approach for the task of human pose transfer, which aims at synthesizing a new image of a person from an input image of that person and a target pose. Unlike existing methods, we propose to estimate dense and intrinsic 3D appearance flow to better guide the transfer of pixels between poses. In particular, we wish to generate the 3D flow from just the reference and target poses. Training a network for this purpose is non-trivial, especially when the annotations for 3D appearance flow are scarce by nature. We address this problem through a flow synthesis stage. This is achieved by fitting a 3D model to the given pose pair and project them back to the 2D plane to compute the dense appearance flow for training. The synthesized ground-truths are then used to train a feedforward network for efficient mapping from the input and target skeleton poses to the 3D appearance flow. With the appearance flow, we perform feature warping on the input image and generate a photorealistic image of the target pose. Extensive results on DeepFashion and Market-1501 datasets demonstrate the effectiveness of our approach over existing methods. Our code is available at

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Dense Intrinsic Appearance Flow for Human Pose Transfer

Huang

Loy

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

216

144

View full text Add to dashboard Cite

show abstract

“…Since current statistical models can not represent clothing, most works [7,26,40,68,48,32,22,67,31,50,8,33,44,46] are restricted to inferring body shape alone. Model fits have been used to virtually dress and manipulate people's shape and clothing in images [50,67,62,36]. None of these approaches recover 3D clothing.…”

Section: Related Workmentioning

confidence: 99%

Multi-Garment Net: Learning to Dress 3D People From Images

Bhatnagar

Tiwari

Theobalt³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

349

134

View full text Add to dashboard Cite

We present Multi-Garment Network (MGN), a method to predict body shape and clothing, layered on top of the SMPL [40] model from a few frames (1-8) of a video. Several experiments demonstrate that this representation allows higher level of control when compared to single mesh or voxel representations of shape. Our model allows to predict garment geometry, relate it to the body shape, and transfer it to new body shapes and poses. To train MGN, we leverage a digital wardrobe containing 712 digital garments in correspondence, obtained with a novel method to register a set of clothing templates to a dataset of real 3D scans of people in different clothing and poses. Garments from the digital wardrobe, or predicted by MGN, can be used to dress any body shape in arbitrary poses. We will make publicly available the digital wardrobe, the MGN model, and code to dress SMPL with the garments at [1].

show abstract

“…Thus, UV-GAN [3] utilizes the main axial symmetry of a face by passing an image and its flipped copy to an inpainting ConvNet. The system in [36] estimates a matrix that corresponds to the probabilities of SMPL model vertices to have similar colors, and use it to color vertices with unobserved colors.…”

Section: Related Workmentioning

confidence: 99%

Coordinate-Based Texture Inpainting for Pose-Guided Human Image Generation

Grigorev

Sevastopolsky

Vakhitov

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

141

113

View full text Add to dashboard Cite

We present a new deep learning approach to pose-guided resynthesis of human photographs. At the heart of the new approach is the estimation of the complete body surface texture based on a single photograph. Since the input photograph always observes only a part of the surface, we suggest a new inpainting method that completes the texture of the human body. Rather than working directly with colors of texture elements, the inpainting network estimates an appropriate source location in the input image for each element of the body surface. This correspondence field between the input image and the texture is then further warped into the target image coordinate frame based on the desired pose, effectively establishing the correspondence between the source and the target view even when the pose change is drastic. The final convolutional network then uses the established correspondence and all other available information to synthesize the output image. A fully-convolutional architecture with deformable skip connections guided by the estimated correspondence field is used. We show state-ofthe-art result for pose-guided image synthesis. Additionally, we demonstrate the performance of our system for garment transfer and pose-guided face resynthesis.

show abstract

Human Appearance Transfer

Cited by 107 publications

References 25 publications

Dense Intrinsic Appearance Flow for Human Pose Transfer

Dense Intrinsic Appearance Flow for Human Pose Transfer

Multi-Garment Net: Learning to Dress 3D People From Images

Coordinate-Based Texture Inpainting for Pose-Guided Human Image Generation

Contact Info

Product

Resources

About