Tensor-based Subspace Factorization for StyleGAN

Haas, René; GraBhof, Stella; Brandt, Sami S.

doi:10.1109/fg52635.2021.9666953

Cited by 3 publications

(1 citation statement)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A popular line of GAN-based image editing research concerns itself with learning so-called "interpretable directions" in the generator's latent space [20,49,48,53,52,57,21,18,19]. Once discovered, such representations of high-level concepts can be manipulated to bring about predictable changes to the images.…”

Section: Introductionmentioning

confidence: 99%

PandA: Unsupervised Learning of Parts and Appearances in the Feature Maps of GANs

Oldfield¹,

Tzelepis²,

Panagakis³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent advances in the understanding of Generative Adversarial Networks (GANs) have led to remarkable progress in visual editing and synthesis tasks, capitalizing on the rich semantics that are embedded in the latent spaces of pre-trained GANs. However, existing methods are often tailored to specific GAN architectures and are limited to either discovering global semantic directions that do not facilitate localized control, or require some form of supervision through manually provided regions or segmentation masks. In this light, we present an architecture-agnostic approach that jointly discovers factors representing spatial parts and their appearances in an entirely unsupervised fashion. These factors are obtained by applying a semi-nonnegative tensor factorization on the feature maps, which in turn enables context-aware local image editing with pixel-level control. In addition, we show that the discovered appearance factors correspond to saliency maps that localize concepts of interest, without using any labels. Experiments on a wide range of GAN architectures and datasets show that, in comparison to the state of the art, our method is far more efficient in terms of training time and, most importantly, provides much more accurate localized control. Our code is available at: https://github.com/james-oldfield/PandA. Figure 1: We propose an unsupervised method for learning a set of factors that correspond to interpretable parts and appearances in a dataset of images. These can be used for multiple tasks: (a) local image editing, (b) context-aware object removal, and (c) producing saliency maps for learnt concepts of interest. Preprint. Under review.

show abstract