Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation

Abstract

Controllable semantic image editing enables a user to change entire image attributes with few clicks, e.g., gradually making a summer scene look like it was taken in winter. Classic approaches for this task use a Generative Adversarial Net (GAN) to learn a latent space and suitable latent-space transformations. However, current approaches often suffer from attribute edits which are entangled, global image identity changes, and diminished photo-realism. To address these concerns, we learn multiple attribute transformations simultaneously, we integrate attribute regression into the training of transformation functions, apply a content loss and an adversarial loss that encourage the maintenance of image identity and photo-realism. We propose quantitative evaluation strategies for measuring controllable editing performance, unlike prior work which primarily focuses on qualitative evaluation. Our model permits better control for both single-and multiple-attribute editing, while also preserving image identity and realism during transformation. We provide empirical results for both real and synthetic images, highlighting that our model achieves state-of-the-art performance for targeted image manipulation.

A review of Generative Adversarial Networks (GANs) and its applications in a wide variety of disciplines -- From Medical to Remote Sensing

“…Image modification could involve modifying or changing some aspects of an image such as changing the hair color, adding a smile, etc. which was demonstrated by ( [131], [199]).…”

Discovering Multiple and Diverse Directions for Cognitive Image Properties

“…The manipulation directions can also be found conditionally using a neural network. [37] uses local transformation to find a direction conditioned by the latent vector. [30] uses a warping network to learn nonlinear paths in the latent space.…”

Controllable facial attribute editing via Gaussian mixture model disentanglement

“…They argue that the subspace after projection can represent some semantic information with respect to facial attributes, and the semantic information learned by GANs can be reused to reasonably control the image generation process. GANSpace [4] performed principal component analysis (PCA) on the latent variables sampled [4], SeFa [5], InterFaceGAN [3], EYE [6] and the proposed method. Each column represents the results of manipulating a specific attribute by different methods.…”

“…InterFaceGAN [3] utilized support vector machines to learn the boundary hyperplanes of different semantic attributes with the supervision of pre-defined attribute tags, then the facial attributes can be edited by moving along the normal vectors of the hyperplanes. Enjoy Your Editing (EYE) [6] can edit the facial attributes by manipulating the column of a matrix, which integrates the directions of all attributes. However, due to the high-dimensional and nonlinear entangling property, the above methods based on linear projection are difficult to disentangle complex semantic attributes, as shown in Fig.…”