Neural Hair Rendering

Chai, Menglei; Ren, Jinchang; Tulyakov, Sergey

doi:10.1007/978-3-030-58523-5_22

Cited by 17 publications

(11 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Video-to-Video Generation. Existing works on video-tovideo generation can synthesize high-quality human motion videos using conditional image generation [7,20,64]. Chan et al [8] apply pre-computed human poses from driving videos as input for novel view and pose generation of a target person.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Flow Guided Transformable Bottleneck Networks for Motion Retargeting

Ren

Chai

Woodford

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

Human motion retargeting aims to transfer the motion of one person in a "driving" video or set of images to another person. Existing efforts leverage a long training video from each target person to train a subject-specific motion transfer model. However, the scalability of such methods is limited, as each model can only generate videos for the given target subject, and such training videos are labor-intensive to acquire and process. Few-shot motion transfer techniques, which only require one or a few images from a target, have recently drawn considerable attention. Methods addressing this task generally use either 2D or explicit 3D representations to transfer motion, and in doing so, sacrifice either accurate geometric modeling or the flexibility of an end-to-end learned representation. Inspired by the Transformable Bottleneck Network, which renders novel views and manipulations of rigid objects, we propose an approach based on an implicit volumetric representation of the image content, which can then be spatially manipulated using volumetric flow fields. We address the challenging question of how to aggregate information across different body poses, learning flow fields that allow for combining content from the appropriate regions of input images of highly non-rigid human subjects performing complex motions into a single implicit volumetric representation. This allows us to learn our 3D representation solely from videos of moving people. Armed with both 3D object understanding and end-to-end learned rendering, this categorically novel representation delivers state-of-the-art image generation quality, as shown by our quantitative and qualitative evaluations.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Many approaches to this task learn to render a specific person [1,7,8,13,20,26,49,64,63,66,67,73] conditioned on the desired pose. This requires a large number of training frames of that person, and incurs substantial training time that must be repeated per each new subject.…”

Section: Introductionmentioning

confidence: 99%

Flow Guided Transformable Bottleneck Networks for Motion Retargeting

Ren

Chai

Woodford

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Hair is a critical component of human portraits and yet is challenging to analyze and synthesize due to its intricate structure and severe self-occlusions therein. Various techniques have been proposed for hair manipulation, such as interactive hair shape editing [2], hair transfer [1], morphing [7], neural hair rendering [8,9], and manipulation of multiple attributes of hair in a single image [3]. However, most of them are based on a coarse 2D orientation map to represent the hair structure, which lacks temporal coherence and fine-grained details, negatively affecting visual quality for video manipulation.…”

Section: Related Workmentioning

confidence: 99%

“…However, most of them are based on a coarse 2D orientation map to represent the hair structure, which lacks temporal coherence and fine-grained details, negatively affecting visual quality for video manipulation. Although Chai et al [9] calculated a warping field to maintain temporal coherence, their method cannot generalize to arbitrary hairstyles due to the dependence on 3D hair models.…”

Section: Related Workmentioning

confidence: 99%

iHairRecolorer: deep image-to-video hair color transfer

Yang

et al. 2021

Sci. China Inf. Sci.

View full text Add to dashboard Cite

“…Generative adversarial networks (GANs), which synthesize images by adversarial training [21], have witnessed tremendous progress in generating high-quality, high-resolution, and photo-realistic images and videos [5,33,68]. In conditional setting [54], the generation process is controlled via additional input signals, such as segmentation information [8,58,60,71,72], class labels [83], and sketches [29,85]. These techniques have seen applications in commercial image editing tools.…”

Section: Introductionmentioning

confidence: 99%

Teachers Do More Than Teach: Compressing Image-to-Image Models

Jin¹,

Ren²,

Woodford³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have achieved huge success in generating high-fidelity images, however, they suffer from low efficiency due to tremendous computational cost and bulky memory usage. Recent efforts on compression GANs show noticeable progress in obtaining smaller generators by sacrificing image quality or involving a time-consuming searching process. In this work, we aim to address these issues by introducing a teacher network that provides a search space in which efficient network architectures can be found, in addition to performing knowledge distillation. First, we revisit the search space of generative models, introducing an inception-based residual block into generators. Second, to achieve target computation cost, we propose a one-step pruning algorithm that searches a student architecture from the teacher model and substantially reduces searching cost. It requires no 1 sparsity regularization and its associated hyper-parameters, simplifying the training procedure. Finally, we propose to distill knowledge through maximizing feature similarity between teacher and student via an index named Global Kernel Alignment (GKA). Our compressed networks achieve similar or even better image fidelity (FID, mIoU) than the original models with much-reduced computational cost, e.g., MACs. Code will be released at https://github.com/snap-research/CAT.

show abstract

Neural Hair Rendering

Cited by 17 publications

References 58 publications

Flow Guided Transformable Bottleneck Networks for Motion Retargeting

Flow Guided Transformable Bottleneck Networks for Motion Retargeting

iHairRecolorer: deep image-to-video hair color transfer

Teachers Do More Than Teach: Compressing Image-to-Image Models

Contact Info

Product

Resources

About