FaceTuneGAN: Face autoencoder for convolutional expression transfer using neural generative adversarial networks

Olivier, Nicolas; Baert, Kelian; Danieau, Fabien; Multon, Franck; Avril, Quentin

doi:10.1016/j.cag.2022.12.004

Cited by 13 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This pioneering work was followed by many other autoencoders which differed from one another mostly by their application domain and the mesh operators used in their architecture [ LBBM18 , FKD*20 , YFST18 , ZWL*20 , GCBZ19 , DS19 , TZY*22 , BBP*19 ]. These mesh operators were used also for generative models based on GAN architectures [ OBD*21 , CBZ*19 ], but they appear to be less frequent than their VAE counterparts. Most GAN architectures operate in the image domain by representing 3D shapes in a UV space [ MPN*20 , LBZ*20 ].…”

Section: Related Workmentioning

confidence: 99%

“…However, [ AW20 ] is specifically designed for grid‐structured data, like images, and [ VB20 ] still requires a pre‐trained GAN and two additional networks for disentanglement. In the 3D shapes domain, GAN disentanglement is still researched to control subject poses and expressions [ CTS*21 , OBD*21 ] or object parts [ LLHF21 ]. However, they suffer the same problems described for 3D VAEs: they have complex architectures and do not have control over the generation of local identity attributes.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

3D Generative Model Latent Disentanglement via Local Eigenprojection

Foti

Koo

Stoyanov

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Designing realistic digital humans is extremely complex. Most data‐driven generative models used to simplify the creation of their underlying geometric shape do not offer control over the generation of local shape attributes. In this paper, we overcome this limitation by introducing a novel loss function grounded in spectral geometry and applicable to different neural‐network‐based generative models of 3D head and body meshes. Encouraging the latent variables of mesh variational autoencoders (VAEs) or generative adversarial networks (GANs) to follow the local eigenprojections of identity attributes, we improve latent disentanglement and properly decouple the attribute creation. Experimental results show that our local eigenprojection disentangled (LED) models not only offer improved disentanglement with respect to the state‐of‐the‐art, but also maintain good generation capabilities with training times comparable to the vanilla implementations of the models. Our code and pre‐trained models are available at github.com/simofoti/LocalEigenprojDisentangled.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

3D Generative Model Latent Disentanglement via Local Eigenprojection

Foti

Koo

Stoyanov

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Sci. 2024, 14, 2149 2 of 13 Another significant challenge is the development of target-based algorithms capable of effectively distinguishing identity features from other facial attributes, such as expressions or poses [17]. This development is necessary to create reliable deepfakes that faithfully imitate the face of a reference person.…”

Section: Introductionmentioning

confidence: 99%

Fast Fake: Easy-to-Train Face Swap Model

Walczyna,

Piotrowski

2024

Applied Sciences

View full text Add to dashboard Cite

The proliferation of “Deep fake” technologies, particularly those facilitating face-swapping in images or videos, poses significant challenges and opportunities in digital media manipulation. Despite considerable advancements, existing methodologies often struggle with maintaining visual coherence, especially in preserving background features and ensuring the realistic integration of identity traits. This study introduces a novel face replacement model that leverages a singular framework to address these issues, employing the Adaptive Attentional Denormalization mechanism from FaceShifter and integrating identity features via ArcFace and BiSeNet for enhanced attribute extraction. Key to our approach is the utilization of Fast GAN, optimizing the training efficiency of our model on relatively small datasets. We demonstrate the model’s efficacy in generating convincing face swaps with high fidelity, showcasing a significant improvement in blending identities seamlessly with the original background context. Our findings contribute to visual deepfake generation by enhancing realism and training efficiency but also highlight the potential for applications where authentic visual representation is crucial.

show abstract

Non-corresponding and topology-free 3D face expression transfer

Liu,

Gai,

2024

Vis Comput

View full text Add to dashboard Cite

FaceTuneGAN: Face autoencoder for convolutional expression transfer using neural generative adversarial networks

Cited by 13 publications

References 11 publications

3D Generative Model Latent Disentanglement via Local Eigenprojection

3D Generative Model Latent Disentanglement via Local Eigenprojection

Fast Fake: Easy-to-Train Face Swap Model

Non-corresponding and topology-free 3D face expression transfer

Contact Info

Product

Resources

About