FReeNet: Multi-Identity Face Reenactment

Zhang, Jiangning; Zeng, Xianfang; Wang, Mengmeng; Pan, Yusu; Liu, Liang; Liu, Yong; Ding, Yu; Fan, Changjie

doi:10.1109/cvpr42600.2020.00537

Cited by 96 publications

(54 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar ideas can be found in recent reenactment models such as First Order Motion Model [13], which uses additional decoder refining incoming image features and tricks to mitigate identity gaps when reference and driving persons differ. Evaluation Metrics: FID [14] is one of the popular evaluation metrics for generative models including face reenactment [3,4,15,16]. However, simply using FID is not suited for the evaluation of face reenactment.…”

Section: Related Workmentioning

confidence: 99%

“…As for the references, we used 3 images which are frontal, left-side (yaw −30 • ), and right-side (yaw +30 • ) faces, no matter which sequence to be evaluated. FID [14] is one of the metrics frequently used for the evaluation of generative models including motion transfer task [15,16]. However, it cannot adequately measure the quality of face reenactment.…”

Section: Motion Transfermentioning

confidence: 99%

See 1 more Smart Citation

Thinking the Fusion Strategy of Multi-reference Face Reenactment

Yashima¹,

Narihira²,

Kojima³

2022

Preprint

View full text Add to dashboard Cite

In recent advances of deep generative models, face reenactment -manipulating and controlling human face, including their head movement-has drawn much attention for its wide range of applicability. Despite its strong expressiveness, it is inevitable that the models fail to reconstruct or accurately generate unseen side of the face of a given single reference image. Most of existing methods alleviate this problem by learning appearances of human faces from large amount of data and generate realistic texture at inference time. Rather than completely relying on what generative models learn, we show that simple extension by using multiple reference images significantly improves generation quality. We show this by 1) conducting the reconstruction task on publicly available dataset, 2) conducting facial motion transfer on our original dataset which consists of multi-person's head movement video sequences, and 3) using a newly proposed evaluation metric to validate that our method achieves better quantitative results.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Motion Transfermentioning

confidence: 99%

Thinking the Fusion Strategy of Multi-reference Face Reenactment

Yashima¹,

Narihira²,

Kojima³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Face generation. Recently, some methods can directly generate realistic faces from the latent space, which boosted many applications in face editing [16,17,18] and face superresolution [5]. Tero et al [19] described a progressive growing methodology for the face generation on 1024×1024 resolution from an underlying code.…”

Section: Related Workmentioning

confidence: 99%

SelFSR: Self-Conditioned Face Super-Resolution in the Wild via Flow Field Degradation Network

Zeng¹,

Zhang²,

Liu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In spite of the success on benchmark datasets, most advanced face super-resolution models perform poorly in real scenarios since the remarkable domain gap between the real images and the synthesized training pairs. To tackle this problem, we propose a novel domain-adaptive degradation network for face super-resolution in the wild. This degradation network predicts a flow field along with an intermediate low resolution image. Then, the degraded counterpart is generated by warping the intermediate image. With the preference of capturing motion blur, such a model performs better at preserving identity consistency between the original images and the degraded. We further present the self-conditioned block for super-resolution network. This block takes the input image as a condition term to effectively utilize facial structure information, eliminating the reliance on explicit priors, e.g. facial landmarks or boundary. Our model achieves state-of-the-art performance on both CelebA and real-world face dataset. The former demonstrates the powerful generative ability of our proposed architecture while the latter shows great identity consistency and perceptual quality in realworld images.

show abstract

“…However, the landmarks are person-specific and transfers the identity information along with the pose and expression which leads to poor cross identity reenactment. To address this MarioNETte [2,19] uses a landmark transformer to remove person-specific information but it requires separate hand-crafted data and model design. A few other models [10,6,20] use action units for reenactment as they are not person-specific.…”

Section: Related Workmentioning

confidence: 99%

Single Source One Shot Reenactment using Weighted motion From Paired Feature Points

Tripathy¹,

Kannala²,

Rahtu³

2021

Preprint

View full text Add to dashboard Cite

Image reenactment is a task where the target object in the source image imitates the motion represented in the driving image. One of the most common reenactment tasks is face image animation. The major challenge in the current face reenactment approaches is to distinguish between facial motion and identity. For this reason, the previous models struggle to produce high-quality animations if the driving and source identities are different (cross-person reenactment). We propose a new (face) reenactment model that learns shape-independent motion features in a selfsupervised setup. The motion is represented using a set of paired feature points extracted from the source and driving images simultaneously. The model is generalised to multiple reenactment tasks including faces and non-face objects using only a single source image. The extensive experiments show that the model faithfully transfers the driving motion to the source while retaining the source identity intact.

show abstract

FReeNet: Multi-Identity Face Reenactment

Cited by 96 publications

References 22 publications

Thinking the Fusion Strategy of Multi-reference Face Reenactment

Thinking the Fusion Strategy of Multi-reference Face Reenactment

SelFSR: Self-Conditioned Face Super-Resolution in the Wild via Flow Field Degradation Network

Single Source One Shot Reenactment using Weighted motion From Paired Feature Points

Contact Info

Product

Resources

About