Towards Disentangling Latent Space for Unsupervised Semantic Face Editing

Liu, Kanglin; Cao, Gaofeng; Zhou, Fei; Liu, Bozhi; Duan, Jiang; Qiu, Guoping

doi:10.1109/tip.2022.3142527

Cited by 19 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This metric describes how drastically the image changes when the intermediate latent space

is interpolated along a certain direction, and its small value represents a relatively smooth latent space and low entanglement. Referring to STIA-WO [ 22 ] the PPL value is calculated for the intermediate latent space

with a certain range of a sampling point along its orthogonal semantic attribute direction, instead of randomly sampling two latent spaces

for calculating the PPL value as

…”

Section: Resultsmentioning

confidence: 99%

“…As shown in Figure 2 , when editing along with the gender semantic attribute direction, the faces in red boxes 1 and 2 have added glasses, and the hair color has changed. STGAN-WO [ 22 ] controls image generation at two scales separately in the latent space and achieves disentanglement of image structure from texture semantic attribute using weight orthogonal regularization, but its regularization method limits the weight space in the network; consequently, reduces the quality of the generated images. Chen et al [ 23 , 24 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning

Zhang

Wang

et al. 2023

Sensors

View full text Add to dashboard Cite

Accurate semantic editing of the generated images is extremely important for machine learning and sample enhancement of big data. Aiming at the problem of semantic entanglement in generated image latent space of the StyleGAN2 network, we proposed a generated image editing method based on global-local Jacobi disentanglement. In terms of global disentanglement, we extract the weight matrix of the style layer in the pre-trained StyleGAN2 network; obtain the semantic attribute direction vector by using the weight matrix eigen decomposition method; finally, utilize this direction vector as the initialization vector for the Jacobi orthogonal regularization search algorithm. Our method improves the speed of the Jacobi orthogonal regularization search algorithm with the proportion of effective semantic attribute editing directions. In terms of local disentanglement, we design a local contrast regularized loss function to relax the semantic association local area and non-local area and utilize the Jacobi orthogonal regularization search algorithm to obtain a more accurate semantic attribute editing direction based on the local area prior MASK. The experimental results show that the proposed method achieves SOTA in semantic attribute disentangled metrics and can discover more accurate editing directions compared with the mainstream unsupervised generated image editing methods.

show abstract

“…This metric describes how drastically the image changes when the intermediate latent space

with a certain range of a sampling point along its orthogonal semantic attribute direction, instead of randomly sampling two latent spaces

for calculating the PPL value as

…”

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning

Zhang

Wang

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Much like transferring selection 'styles', this example provides the obvious benefit of not requiring pre-defined population parameters for generating all aspects of the desired training data. Finally, a particularly intriguing use would involve combining work on creating an interpretable latent space, where population parameters can be translated into different parts of a multi-dimensional Gaussian (Liu et al 2022), with the ability to project generated examples onto this latent space (Karras et al 2021). While the latent space interpretation would allow for a simulator that is instantaneous once trained, the combination with latent space projection would make the network capable of inferring the demographic parameters of real data and testing evolutionary hypotheses using the network directly.…”

Section: Discussionmentioning

confidence: 99%

“…Much like transferring selection ‘styles’, this example provides the obvious benefit of not requiring pre-defined population parameters for generating all aspects of the desired training data. Finally, a particularly intriguing use would involve combining work on creating an interpretable latent space, where population parameters can be translated into different parts of a multi-dimensional Gaussian (Liu et al . 2022), with the ability to project generated examples onto this latent space (Karras et al .…”

Section: Discussionmentioning

confidence: 99%

This population does not exist: learning the distribution of evolutionary histories with generative adversarial networks

Booker

Ray

Schrider

2022

Preprint

View full text Add to dashboard Cite

Numerous studies over the last decade have demonstrated the utility of machine learning methods when applied to population genetic tasks. More recent studies show the potential of deep learning methods in particular, which allow researchers to approach problems without making prior assumptions about how the data should be summarized or manipulated, instead learning their own internal representation of the data in an attempt to maximize inferential accuracy. One type of deep neural network, called Generative Adversarial Networks (GANs), can even be used to generate new data, and this approach has been used to create individual artificial human genomes free from privacy concerns. In this study, we further explore the application of GANs in population genetics by designing and training a network to learn the statistical distribution of population genetic alignments (i.e. data sets consisting of sequences from an entire population sample) under several diverse evolutionary histories—the first GAN capable of performing this task. After testing multiple different neural network architectures, we report the results of a fully differentiable Deep-Convolutional Wasserstein GAN with gradient penalty that is capable of generating artificial examples of population genetic alignments that successfully mimic key aspects of the training data, including the site frequency spectrum, differentiation between populations, and patterns of linkage disequilibrium. We demonstrate consistent training success across various evolutionary models, including models of panmictic and subdivided populations, populations at equilibrium and experiencing changes in size, and populations experiencing either no selection or positive selection of various strengths, all without the need for extensive hyperparameter tuning. Overall, our findings highlight the ability of GANs to learn and mimic population genetic data, and suggest future areas where this work can be applied in population genetics research that we discuss herein.AUTHOR SUMMARYThe application of deep-learning to biological problems has expanded greatly over the last decade. One type of deep neural network, called a Generative Adversarial Network (GAN), attempts to generate artificial examples of a given type of data by learning to fool a discriminator that is simultaneously learning to discriminate between real and artificial examples. In this study, we design a GAN whose purpose is to generate artificial examples of genetic alignments from biological populations of varying evolutionary histories—essentially learning the statistical distribution of those evolutionary histories. We show that our GAN is able to mimic key aspects of the genetic alignments relevant to population genetics, and that the GAN does not require extensive tuning of the network parameters. Ultimately, this work demonstrates the ability of these networks to learn and mimic population genetic data, and highlights future areas where this work can be applied and expanded.

show abstract

“…A perfectly disentangled latent representation where each dimension represents a human-understandable concept would naturally be interpretable. However, achieving a fully disentangled representation is not feasible in the general case, as approaches require specific training data [241], supervision [182], or predefined concepts [145]. Nevertheless, there are approaches using SCMs (Section 2.3) to identify confounders and then disentangle representations to produce more interpretable models.…”

Section: Disentangled Representationsmentioning

confidence: 99%

A Review of the Role of Causality in Developing Trustworthy AI Systems

Ganguly¹,

Fazlija²,

Badar³

et al. 2023

Preprint

View full text Add to dashboard Cite

State-of-the-art AI models largely lack an understanding of the cause-effect relationship that governs human understanding of the real world. Consequently, these models do not generalize to unseen data, often produce unfair results, and are difficult to interpret. This has led to efforts to improve the trustworthiness aspects of AI models. Recently, causal modeling and inference methods have emerged as powerful tools. This review aims to provide the reader with an overview of causal methods that have been developed to improve the trustworthiness of AI models. We hope that our contribution will motivate future research on causality-based solutions for trustworthy AI.

show abstract

Towards Disentangling Latent Space for Unsupervised Semantic Face Editing

Cited by 19 publications

References 26 publications

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning

This population does not exist: learning the distribution of evolutionary histories with generative adversarial networks

A Review of the Role of Causality in Developing Trustworthy AI Systems

Contact Info

Product

Resources

About