Face Generation and Editing with StyleGAN: A Survey

Melnik, Andrew; Miasayedzenkau, Maksim; Makarovets, Dzianis; Pirshtuk, Dzianis; Akbulut, Eren; Holzmann, Dennis; Renusch, Tarek; Reichert, Gustav; Ritter, Helge

doi:10.48550/arxiv.2212.09102

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method is similar to OroJaRGAN in terms of PPL metrics because the search method is the same, only the semantic direction is initialized in a different way. Referring to papers [ 42 , 43 ], the equivalent model of SefaGAN considered only that the role of the mapping function in the network is limited to the style layer, ignoring the role of other layers. the effect of PCAGAN depends heavily on the number of sampling points.…”

Section: Resultsmentioning

confidence: 99%

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning

Zhang

Wang

et al. 2023

Sensors

View full text Add to dashboard Cite

Accurate semantic editing of the generated images is extremely important for machine learning and sample enhancement of big data. Aiming at the problem of semantic entanglement in generated image latent space of the StyleGAN2 network, we proposed a generated image editing method based on global-local Jacobi disentanglement. In terms of global disentanglement, we extract the weight matrix of the style layer in the pre-trained StyleGAN2 network; obtain the semantic attribute direction vector by using the weight matrix eigen decomposition method; finally, utilize this direction vector as the initialization vector for the Jacobi orthogonal regularization search algorithm. Our method improves the speed of the Jacobi orthogonal regularization search algorithm with the proportion of effective semantic attribute editing directions. In terms of local disentanglement, we design a local contrast regularized loss function to relax the semantic association local area and non-local area and utilize the Jacobi orthogonal regularization search algorithm to obtain a more accurate semantic attribute editing direction based on the local area prior MASK. The experimental results show that the proposed method achieves SOTA in semantic attribute disentangled metrics and can discover more accurate editing directions compared with the mainstream unsupervised generated image editing methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning

Zhang

Wang

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…All three StyleGAN variants share the same basic architecture and follow the common design of style-based GANs in general (Fig. 1) 17 .…”

Section: Pretrained Generator Modelsmentioning

confidence: 99%

CleftGAN: Adapting A Style-Based Generative Adversarial Network To Create Images Depicting Cleft Lip Deformity

Hayajneh,

Serpedin,

Shaqfeh

et al. 2024

Preprint

View full text Add to dashboard Cite

Background Training a machine learning system to evaluate any type of facial deformity is impeded by the scarcity of large datasets of high-quality, ethics board-approved patient images. We have built a deep learning-based cleft lip generator called CleftGAN designed to produce an almost unlimited number of high-fidelity facsimiles of cleft lip facial images with wide variation. Methods A transfer learning protocol testing different versions of StyleGAN as the base model was undertaken. Data augmentation maneuvers permitted input of merely 514 frontal photographs of cleft-affected faces adapted to a base model of 70,000 normal faces. The Frechet Inception Distance was used to measure the similarity of the newly generated facial images to the cleft training dataset. Perceptual Path Length and the novel Divergence Index of Normality measures also assessed the performance of the novel image generator. Results CleftGAN generates vast numbers of unique faces depicting a wide range of cleft lip deformity with variation of ethnic background. Performance metrics demonstrated a high similarity of the generated images to our training dataset and a smooth, semantically valid interpolation of images through the transfer learning process. The distribution of normality for the training and generated images were highly comparable. Conclusions CleftGAN is a novel instrument that generates an almost boundless number of realistic facial images depicting cleft lip. This tool promises to become a valuable resource for the development of machine learning models to objectively evaluate facial form and the outcomes of surgical reconstruction.

show abstract

“…StyleGAN [28], [46] excels at producing high-fidelity facial images with fine-grained control over hierarchical channel styles. Many techniques leverage StyleGAN for the generation of high-quality images and the manipulation of facial characteristics, as well as for various applications related to faces, including swapping, restoration, de-aging, and reenactment [47], [48]. Pinkney and Adler [49] further enhanced StyleGAN using sparse cartoon data, demonstrating its effectiveness in generating lifelike cartoon faces.…”

Section: B Style Transfermentioning

confidence: 99%

High-Quality Face Caricature via Style Translation

Laishram,

Shaheryar,

Lee

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Caricature is an exaggerated form of artistic portraiture that accentuates unique yet subtle characteristics of human faces. Recently, advancements in deep end-to-end techniques have yielded encouraging outcomes in capturing both style and elevated exaggerations in creating face caricatures. The majority of these approaches tend to produce cartoon-like results that are impractical for real-world applications. In this study, we proposed a high-quality, unpaired face caricature method that is appropriate for use in the real world and uses computer vision techniques and GAN models. We attain the exaggeration of facial features and the stylization of appearance through a two-step process: Face caricature generation and face caricature projection. The face caricature generation step creates new caricature face datasets from real images and trains a generative model using the real and newly created caricature datasets. The Face caricature projection employs an encoder, which is trained with real and caricature faces with the pretrained generator to project real and caricature faces. Using the encoder and generator's latent space, we perform an iterative facial exaggeration from the real image to the caricature faces. Our projection preserves the facial identity, attributes, and expressions from the input image. Also, it accounts for facial occlusions, such as reading glasses or sunglasses, to enhance the robustness of our model. Furthermore, we conducted a comprehensive comparison of our approach with various state-of-the-art face caricature methods, which highlighted the distinctiveness and exceptional realism of our process.

show abstract

Face Generation and Editing with StyleGAN: A Survey

Cited by 3 publications

References 0 publications

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning

CleftGAN: Adapting A Style-Based Generative Adversarial Network To Create Images Depicting Cleft Lip Deformity

High-Quality Face Caricature via Style Translation

Contact Info

Product

Resources

About