FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery

Singh, Krishna Kumar; Ojha, Utkarsh; Lee, Yong Jae

doi:10.48550/arxiv.1811.11155

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It also injects noise at each level of the generator to capture stochastic variations. Singh et al introduce Fine-GAN [45], a generative model which disentangles the background, object shape, and object appearance to hierarchically generate images of fine-grained object categories. Layered Recursive GAN is proposed in [60], and generates image background and foreground separately and recursively without direct supervision.…”

Section: Fine-grained Image Generationmentioning

confidence: 99%

Fine-grained Synthesis of Unrestricted Adversarial Examples

Poursaeed¹,

Jiang²,

Goshu³

et al. 2019

Preprint

View full text Add to dashboard Cite

We propose a novel approach for generating unrestricted adversarial examples by manipulating fine-grained aspects of image generation. Unlike existing unrestricted attacks that typically hand-craft geometric transformations, we learn stylistic and stochastic modifications leveraging stateof-the-art generative models. This allows us to manipulate an image in a controlled, fine-grained manner without being bounded by a norm threshold. Our model can be used for both targeted and non-targeted unrestricted attacks. We demonstrate that our attacks can bypass certified defenses, yet our adversarial images look indistinguishable from natural images as verified by human evaluation. Adversarial training can be used as an effective defense without degrading performance of the model on clean images. We perform experiments on LSUN and CelebA-HQ as high resolution datasets to validate efficacy of our proposed approach.

show abstract

Section: Fine-grained Image Generationmentioning

confidence: 99%

Fine-grained Synthesis of Unrestricted Adversarial Examples

Poursaeed¹,

Jiang²,

Goshu³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The most similar approach to ours is FineGAN [23], which our generators and discriminators are based upon. However, there are many significant differences and additions: (i) We added a set of encoders which are trained to support new tasks.…”

Section: Related Workmentioning

confidence: 99%

“…Discriminators Following [23], the two discriminators are adversarial opponents on the outputs I bg , I. The background discriminator D bg receives either a real background image from the set X bg , a fake background image generated by the background generator, or a real object image from the set X c .…”

Section: Generatorsmentioning

confidence: 99%

OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clustering

Benny,

Wolf

2019

Preprint

View full text Add to dashboard Cite

We present a method for simultaneously learning, in an unsupervised manner, (i) a conditional image generator, (ii) foreground extraction and segmentation, (iii) clustering into a two-level class hierarchy, and (iv) object removal and background completion, all done without any use of annotation. The method combines a generative adversarial network and a variational autoencoder, with multiple encoders, generators and discriminators, and benefits from solving all tasks at once. The input to the training scheme is a varied collection of unlabeled images from the same domain, as well as a set of background images without a foreground object. In addition, the image generator can mix the background from one image, with a foreground that is conditioned either on that of a second image or on the index of a desired cluster. The method obtains state of the art results in comparison to the literature methods, when compared to the current state of the art in each of the tasks.

show abstract

“…For datasets where the form of the generative model is not know, deep representation learning methods often look for factorial or disentagled representations [7,2,8,3,[9][10][11]. While factorial representations are useful for certain tasks like sampling, they are less useful for understanding datasets with hierarchy and compositionality.…”

Section: Structured Representations In Deep Networkmentioning

confidence: 99%

Hangul Fonts Dataset: a Hierarchical and Compositional Dataset for Investigating Learned Representations

Livezey¹,

Hwang²,

Yeung³

et al. 2019

Preprint

View full text Add to dashboard Cite

Interpretable representations of data are useful for testing a hypothesis or to distinguish between multiple potential hypotheses about the data. In contrast, applied machine learning, and specifically deep learning (DL), is often used in contexts where performance is valued over interpretability. Indeed, deep networks (DNs) are often treated as "black boxes", and it is not well understood what and how they learn from a given dataset. This lack of understanding seriously hinders adoption of DNs as data analysis tools in science and poses numerous research questions. One problem is that current deep learning research datasets either have very little hierarchical structure or are too complex for their structure to be analyzed, impeding precise predictions of hierarchical representations. To address this gap, we present a benchmark dataset with known hierarchical and compositional structure and a set of methods for performing hypothesis-driven data analysis using DNs. The Hangul Fonts Dataset is composed of 35 fonts, each with 11,172 written syllables consisting of 19 initial consonants, 21 medial vowels, and 28 final consonants. The rules for combining and modifying individual Hangul characters into blocks can be encoded, with translation, scaling, and style variation that depend on precise block content, as well as naturalistic variation across fonts. Thus, the Hangul Fonts Dataset will provide an intermediate complexity dataset with well-defined, hierarchical features to interrogate learned representations. We first present a summary of the structure of the dataset. Using a set of unsupervised and supervised methods, we find that deep network representations contain structure related to the geometrical hierarchy of the characters. Our results lay the foundation for a better understanding of what deep networks learn from complex, structured datasets.Preprint. Under review.

show abstract

FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery

Cited by 3 publications

References 0 publications

Fine-grained Synthesis of Unrestricted Adversarial Examples

Fine-grained Synthesis of Unrestricted Adversarial Examples

OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clustering

Hangul Fonts Dataset: a Hierarchical and Compositional Dataset for Investigating Learned Representations

Contact Info

Product

Resources

About