Learning What and Where to Draw

Reed, Scott; Akata, Zeynep; Mohan, Santosh; Tenka, Samuel; Schiele, Bernt; Lee, Honglak

doi:10.48550/arxiv.1610.02454

Cited by 9 publications

(9 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Over the next few years, progress continued using a combination of methods. These include improving the generative model architecture with modifications like multi-scale generators (Zhang et al, 2017;, integrating attention and auxiliary losses (Xu et al, 2018), and leveraging additional sources of conditioning information beyond just text (Reed et al, 2016a;Li et al, 2019;Koh et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Zero-Shot Text-to-Image Generation

Ramesh,

Pavlov,

Goh

et al. 2021

Preprint

272

409

View full text Add to dashboard Cite

Text-to-image generation has traditionally focused on finding better modeling assumptions for training on a fixed dataset. These assumptions might involve complex architectures, auxiliary losses, or side information such as object part labels or segmentation masks supplied during training. We describe a simple approach for this task based on a transformer that autoregressively models the text and image tokens as a single stream of data. With sufficient data and scale, our approach is competitive with previous domain-specific models when evaluated in a zero-shot fashion.

show abstract

Section: Introductionmentioning

confidence: 99%

Zero-Shot Text-to-Image Generation

Ramesh,

Pavlov,

Goh

et al. 2021

Preprint

272

409

View full text Add to dashboard Cite

show abstract

“…In recent times, GAN-based approaches have been used to generate impressively realistic house-numbers [4], faces, bedrooms [17] and a variety of other image categories [18,21]. Usually, these image categories tend to have extremely complex underlying distributions.…”

Section: Introductionmentioning

confidence: 99%

DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data

Swaminathan

Sarvadevabhatla

Babu

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

243

157

View full text Add to dashboard Cite

A class of recent approaches for generating images, called Generative Adversarial Networks (GAN), have been used to generate impressively realistic images of objects, bedrooms, handwritten digits and a variety of other image modalities. However, typical GAN-based approaches require large amounts of training data to capture the diversity across the image modality. In this paper, we propose DeLiGAN -a novel GAN-based architecture for diverse and limited training data scenarios. In our approach, we reparameterize the latent generative space as a mixture model and learn the mixture model's parameters along with those of GAN. This seemingly simple modification to the GAN framework is surprisingly effective and results in models which enable diversity in generated samples although trained with limited data. In our work, we show that DeLi-GAN can generate images of handwritten digits, objects and hand-drawn sketches, all using limited amounts of data. To quantitatively characterize intra-class diversity of generated samples, we also introduce a modified version of "inception-score", a measure which has been found to correlate well with human assessment of generated samples.

show abstract

“…However, their training is notoriously difficult and requires much effort to create any satisfactory results. GANs take as an input a vector, which might be random, fixed, or generated from the text [29,30,31].…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

Level generation and style enhancement -- deep learning for game development overview

Migdał,

Olechno,

Podgórski

2021

Preprint

View full text Add to dashboard Cite

We present practical approaches of using deep learning to create and enhance level maps and textures for video games -desktop, mobile, and web. We aim to present new possibilities for game developers and level artists.The task of designing levels and filling them with details is challenging. It is both time-consuming and takes effort to make levels rich, complex, and with a feeling of being natural. Fortunately, recent progress in deep learning provides new tools to accompany level designers and visual artists. Moreover, they offer a way to generate infinite worlds for game replayability and adjust educational games to players' needs.We present seven approaches to create level maps, each using statistical methods, machine learning, or deep learning. In particular, we include:

show abstract

Learning What and Where to Draw

Cited by 9 publications

References 5 publications

Zero-Shot Text-to-Image Generation

Zero-Shot Text-to-Image Generation

DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data

Level generation and style enhancement -- deep learning for game development overview

Contact Info

Product

Resources

About