Labels4Free: Unsupervised Segmentation using StyleGAN

Abdal, Rameen; Zhu, Peihao; Mitra, Niloy J.; Wonka, Peter

doi:10.1109/iccv48922.2021.01371

Cited by 61 publications

(30 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once both manipulations converge to a single viewpoint, the authors can employ the STN to align real images. Abdal et al [2021b] present an unsupervised segmentation method based on a pre-trained GAN. The authors recognize that nullifying some of the activations actually causes the GAN to erase the foreground object, producing an image with only a background.…”

Section: Discriminative Applicationsmentioning

confidence: 99%

State-of-the-Art in the Architecture, Methods and Applications of StyleGAN

Bermano¹,

Gal²,

Alaluf³

et al. 2022

Preprint

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have established themselves as a prevalent approach to image synthesis. Of these, StyleGAN offers a fascinating case study, owing to its remarkable visual quality and an ability to support a large array of downstream tasks. This state-of-the-art report covers the StyleGAN architecture, and the ways it has been employed since its conception, while also analyzing its severe limitations. It aims to be of use for both newcomers, who wish to get a grasp of the field, and for more experienced readers that might benefit from seeing current research trends and existing tools laid out. Among StyleGAN's most interesting aspects is its learned latent space. Despite being learned with no supervision, it is surprisingly well-behaved and remarkably disentangled. Combined with StyleGAN's visual quality, these properties gave rise to unparalleled editing capabilities. However, the control offered by StyleGAN is inherently limited to the generator's learned distribution, and can only be applied to images generated by StyleGAN itself. Seeking to bring StyleGAN's latent control to real-world scenarios, the study of GAN inversion and latent space embedding has quickly gained in popularity. Meanwhile, this same study has helped shed light on the inner workings and limitations of StyleGAN. We map out StyleGAN's impressive story through these investigations, and discuss the details that have made StyleGAN the go-to generator. We further elaborate on the visual priors StyleGAN constructs, and discuss their use in downstream discriminative tasks. Looking forward, we point out StyleGAN's limitations and speculate on current trends and promising directions for future research, such as task and target specific fine-tuning.

show abstract

Section: Discriminative Applicationsmentioning

confidence: 99%

State-of-the-Art in the Architecture, Methods and Applications of StyleGAN

Bermano¹,

Gal²,

Alaluf³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Among various types of generative models, such as variational auto-encoder (VAEs) [16,28,46,55], flow-based model [27,47], diffusion model [10,17], etc., GAN [11] has received wide attention due to its impressive performance on both unconditional synthesis [22,[24][25][26] and conditional synthesis [6,36,49,64]. Early studies on interpreting GANs [5,8,51,62] suggest that, a well-learned GAN generator has encoded rich knowledge that can be promising applied to various downstream tasks, including attribute editing [2,3,20,33,62,63,67], image processing [12,18,40,48,68], superresolution [7,35], image classification [61], semantic segmentation [1,32,54,60,65], and visual alignment [44]. Existing interpretation approaches usually focus on the relationship between the latent space and the image space [51,58,62,…”

Section: Related Workmentioning

confidence: 99%

“…This work clearly differs from prior arts from the following aspects. (1) We inspect the conditional generator from the channel perspective, which aggregates the messages from both the latent code and the class embedding. To our knowledge, this is the first attempt on understanding the function of embedding space in conditional generation.…”

Section: Related Workmentioning

confidence: 99%

“…Existing works [1,32,33,54,60,65] revealed that internal features in GAN can enable object part segmentation of generated images, facilitating semantic annotation synthesis and local semantic editing. Here, we present a simple prototype that utilizes category-oriented channel awareness to do unsupervised semantic segmentation on generated images.…”

Section: Segmentation Via Channel Weighted Clusteringmentioning

confidence: 99%

“…We further verify that the proposed channel awareness shows promising potential in (3) segmenting the synthesized image and (4) evaluating the category-wise synthesis performance. 1…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Interpreting Class Conditional GANs with Channel Awareness

He¹,

Zhang²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Unsupervised Domain Adaptation for Simultaneous Segmentation and Classification of the Retinal Arteries and Veins

Xue,

Zhang,

et al. 2024

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Automatic segmentation of the fundus retinal vessels and accurate classification of the arterial and venous vessels play an important role in clinical diagnosis. This article proposes a fundus retinal vascular segmentation and arteriovenous classification network that combines the adversarial training and attention mechanism to address the issues of fundus retinal arteriovenous classification error and ambiguous segmentation of fine blood vessels. It consists of three core components: discriminator, generator, and segmenter. In order to address the domain shift issue, U‐Net is employed as a discriminator, and data samples for arterial and venous vessels are generated with a generator using an unsupervised domain adaption (UDA) approach. The classification of retinal arterial and venous vessels (A/V) as well as the segmentation of fine vessels is improved by adding a self‐attention mechanism to improve attention to vessel edge features and the terminal fine vessels. Non‐strided convolution and non‐pooled downsampling methods are also used to avoid losing fine‐grained information and learning less effective feature representations. The performance of multi‐class blood vessel segmentation is as follows, per test results on the DRIVE dataset: F1‐score (F1) has a value of 0.7496 and an accuracy of 0.9820. The accuracy of A/V categorization has increased by 1.35% when compared to AU‐Net. The outcomes demonstrate that by enhancing the baseline U‐Net, the strategy we suggested enhances the automated classification and segmentation of blood vessels.

show abstract

Labels4Free: Unsupervised Segmentation using StyleGAN

Cited by 61 publications

References 18 publications

State-of-the-Art in the Architecture, Methods and Applications of StyleGAN

State-of-the-Art in the Architecture, Methods and Applications of StyleGAN

Interpreting Class Conditional GANs with Channel Awareness

Unsupervised Domain Adaptation for Simultaneous Segmentation and Classification of the Retinal Arteries and Veins

Contact Info

Product

Resources

About