Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval

Chong, Min Jin; Chu, Wen-Sheng; Kumar, Abhishek; Forsyth, David

doi:10.1109/iccv48922.2021.00386

Cited by 24 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ipants 4 . As can be seen from the results in Table 2, our method showed significantly better diversity than [34] with a p-value of < 0.0001.…”

Section: Comparison With Supervised Methodsmentioning

confidence: 99%

“…Given a reference image and a target image, they exchange style codes based on regional differences to transfer the appearance of an object. [4] improves upon [5] by finding more successful image-specific manipulation directions and eliminating the per-image matching overhead. [20] clusters the feature maps to find meaningful and interpretable semantic classes that can be used to create segmentation masks.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs

Simsar¹,

Kocasari²,

Er³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…ipants 4 . As can be seen from the results in Table 2, our method showed significantly better diversity than [34] with a p-value of < 0.0001.…”

Section: Comparison With Supervised Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs

Simsar¹,

Kocasari²,

Er³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To reduce supervision, others have explored both unsupervised approaches [8,11,26,61,70] and self-supervised approaches [29,53,69]. To achieve more fine-grained control, many works have explored the mixing of latent codes [14,16,30,62] and localbased editing via semantic maps [42,82] or reference images [37,40]. Finally, to achieve text-based editing some have leveraged powerful contrast language-image (CLIP) models [3,9,12,21,48].…”

Section: A2 Editing Images With Stylegan2mentioning

confidence: 99%

Third Time's the Charm? Image and Video Editing with StyleGAN3

Alaluf¹,

Patashnik²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

StyleGAN is arguably one of the most intriguing and well-studied generative models, demonstrating impressive performance in image generation, inversion, and manipulation. In this work, we explore the recent StyleGAN3 architecture, compare it to its predecessor, and investigate its unique advantages, as well as drawbacks. In particular, we demonstrate that while StyleGAN3 can be trained on unaligned data, one can still use aligned data for training, without hindering the ability to generate unaligned imagery. Next, our analysis of the disentanglement of the different latent spaces of StyleGAN3 indicates that the commonly used W/W+ spaces are more entangled than their StyleGAN2 counterparts, underscoring the benefits of using the StyleSpace for fine-grained editing. Considering image inversion, we observe that existing encoder-based techniques struggle when trained on unaligned data. We therefore propose an encoding scheme trained solely on aligned data, yet can still invert unaligned images. Finally, we introduce a novel video inversion and editing workflow that leverages the capabilities of a fine-tuned Style-GAN3 generator to reduce texture sticking and expand the field of view of the edited video. Code is available on our project page: https://yuval-alaluf.github.io/ stylegan3-editing/.* Denotes equal contribution.

show abstract

“…person identity). Hou et al [2022], , Tewari et al [2020a,b], focus on searching latent space to find latent codes corresponding to global meaningful manipulations, while Chong et al [2021] utilizes semantic segmentation maps to locate and mix certain positions of style codes to achieve editing goals.…”

Section: Stylegan-based Editingmentioning

confidence: 99%

Grasping the Arrow of Time from the Singularity: Decoding Micromotion in Low-dimensional Latent Spaces from StyleGAN

Wu¹,

Jiang²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

The disentanglement of StyleGAN latent space has paved the way for realistic and controllable image editing, but does StyleGAN know anything about temporal motion, as it was only trained on static images? To study the motion features in the latent space of StyleGAN, in this paper, we hypothesize and demonstrate that a series of meaningful, natural, and versatile small, local movements (referred to as "micromotion", such as expression, head movement, and aging effect) can be represented in low-rank spaces extracted from the latent space of a conventionally pre-trained StyleGAN-v2 model for face generation, with the guidance of proper "anchors" in the form of either short text or video clips. Starting from one target face image, with the editing direction decoded from the low-rank space, its micromotion features can be represented as simple as an affine transformation over its latent feature. Perhaps more surprisingly, such micromotion subspace, even learned from just single target face, can be painlessly transferred to other unseen face images, even those from vastly different domains (such as oil painting, cartoon, and sculpture faces). It demonstrates that the local feature geometry corresponding to one type of micromotion is aligned across different face subjects, and hence that StyleGAN-v2 is indeed "secretly" aware of the subject-disentangled feature variations caused by that micromotion. We present various successful examples of applying our low-dimensional micromotion subspace technique to directly and effortlessly manipulate faces, showing high robustness, low computational overhead, and impressive domain transferability. Our codes are available at https: //github.com/wuqiuche/micromotion-StyleGAN.

show abstract

Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval

Cited by 24 publications

References 26 publications

Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs

Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs

Third Time's the Charm? Image and Video Editing with StyleGAN3

Grasping the Arrow of Time from the Singularity: Decoding Micromotion in Low-dimensional Latent Spaces from StyleGAN

Contact Info

Product

Resources

About