Editing in Style: Uncovering the Local Semantics of GANs

Collins, Edo; Bala, Raja; Price, Bob; Süsstrunk, Sabine

doi:10.1109/cvpr42600.2020.00581

Cited by 283 publications

(236 citation statements)

References 13 publications

Supporting

Mentioning

236

Contrasting

Order By: Relevance

“…Conditional GAN networks [Chakraborty et al 2020;Mirza and Osindero 2014] and GAN editing and inversion methods [Abdal et al 2020;Alharbi and Wonka 2020;Collins et al 2020;Dorta et al 2020;Huang et al 2020;Pinkney and Adler 2020;Richardson et al 2020;Zhu et al 2020] are also related to our method. Alharbi and Wonka [2020] uses a grid structure to inject noise into a GAN to achieve spatial disentanglement on a grid, and then edit the image.…”

Section: Related Workmentioning

confidence: 98%

“…Alharbi and Wonka [2020] uses a grid structure to inject noise into a GAN to achieve spatial disentanglement on a grid, and then edit the image. Collins et al [2020] further accounts for spatial semantics by using Kmeans clustering to calculate spatial overlap between the StyleGAN activation tensors and semantic regions of an image. Collins et al [2020] then uses a greedy algorithm to find interpolation coefficients that maximize changes within a semantic region of interest while minimizing changes outside of the region of interest.…”

Section: Related Workmentioning

confidence: 99%

“…Per layer optimization enables semantically meaningful and high quality results. Unlike previous general GAN editing methods [Alharbi and Wonka 2020;Collins et al 2020], which require manual choice of noise injection structure or manual identification of clusters and fixed parameters for all layers, our method automatically computes the best interpolation coefficients per layer by optimizing a loss function designed to preserve the identity and body of the person while warping only the garment. Results section shows comparison to Collins et al [2020] and the importance of per layer optimization.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TryOnGAN

Lewis

Varadharajan

Kemelmacher-Shlizerman

2021

ACM Trans. Graph.

View full text Add to dashboard Cite

Given a pair of images---target person and garment on another person---we automatically generate the target person in the given garment. Previous methods mostly focused on texture transfer via paired data training, while overlooking body shape deformations, skin color, and seamless blending of garment with the person. This work focuses on those three components, while also not requiring paired data training. We designed a pose conditioned StyleGAN2 architecture with a clothing segmentation branch that is trained on images of people wearing garments. Once trained, we propose a new layered latent space interpolation method that allows us to preserve and synthesize skin color and target body shape while transferring the garment from a different person. We demonstrate results on high resolution 512 × 512 images, and extensively compare to state of the art in try-on on both latent space generated and real images.

show abstract

Section: Related Workmentioning

confidence: 98%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

TryOnGAN

Lewis

Varadharajan

Kemelmacher-Shlizerman

2021

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…Abdal et al [16] learn a translation between vectors in W+, changing a collection of fixed named attributes. Finally, By modifying relevant components of the latent code, Collins et al [17] performs local semantic editing.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…For several image-to-image activities that convert between two separate data realms, such as segmentation charts, drawings, margins, etc., this constraint is a significant restriction. By optimizing an expanded room, such as W+, this restriction can be bypassed [16,17]. In this case, though, the latent space doesn't contain rich semantics for an ambiguous data area.…”

Section: Feature Disentanglementmentioning

confidence: 99%

Facade Style Mixing using Artificial Intelligence for Urban Infill

Ali¹,

Oj²

2021

Preprint

View full text Add to dashboard Cite

Artificial Intelligence and especially machine learning have noticed rapid advancement on image processing operations. However, its involvement in the architectural design is still in its initial stages compared to other disciplines. Therefore, this paper addresses the issues of developing an integrated bottom up digital design approach and details a research framework for the incorporation of Deep Convolutional Generative Adversarial Network (GAN) for early stage design exploration and generation of intricate and complex alternative facade designs for urban infill. This paper proposes a novel building facade design by merging two neighboring building’s architecture style, size, scale, openings, as reference to create a new building design in the same neighborhood for urban infill. This newly produced building contains the outline, style and shape of the parent buildings. A 2D urban infill building design is generated as a picture where 1) neighboring buildings are imported as a reference using mobile phone and 2)iFACADE decode their spatial adjacency. It is depicted the iFACADE will be useful for designers in the early design stage to generate new façades depending on existing buildings in a short time that will save time and energy. Besides, building owners can use iFACADE to show their architects their preferred architecture facade by mixing two building styles and generating a new building. Therefore, it is depicted that iFACADE can become a communication platform in the early design stages between architects and owners. Initial results properly define a heuristic function for generating abstract design facade elements and sufficiently illustrate the desired functionality of our developed prototype.

show abstract

Controlling StyleGANs using rough scribbles via one‐shot learning

Endo

Kanamori

2022

Computer Animation & Virtual

View full text Add to dashboard Cite

This paper tackles the challenging problem of one-shot semantic image synthesis from rough sparse annotations, which we call "semantic scribbles." Namely, from only a single training pair annotated with semantic scribbles, we generate realistic and diverse images with layout control over, for example, facial part layouts and body poses. We present a training strategy that performs pseudo labeling for semantic scribbles using the StyleGAN prior. Our key idea is to construct a simple mapping between StyleGAN features and each semantic class from a single example of semantic scribbles. With such mappings, we can generate an unlimited number of pseudo semantic scribbles from random noise to train an encoder for controlling a pretrained StyleGAN generator. Even with our rough pseudo semantic scribbles obtained via one-shot supervision, our method can synthesize high-quality images thanks to our GAN inversion framework.We further offer optimization-based postprocessing to refine the pixel alignment of synthesized images. Qualitative and quantitative results on various datasets demonstrate improvement over previous approaches in one-shot settings.

show abstract

Editing in Style: Uncovering the Local Semantics of GANs

Cited by 283 publications

References 13 publications

TryOnGAN

TryOnGAN

Facade Style Mixing using Artificial Intelligence for Urban Infill

Controlling StyleGANs using rough scribbles via one‐shot learning

Contact Info

Product

Resources

About