Adversarial Generation of Natural Language

Abstract

Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation. Advances in the adversarial generation of natural language from noise however are not commensurate with the progress made in generating images, and still lag far behind likelihood based methods. In this paper, we take a step towards generating natural language with a GAN objective alone. We introduce a simple baseline that addresses the discrete output space problem without relying on gradient estimators and show that it is able to achieve state-of-the-art results on a Chinese poem generation dataset. We present quantitative results on generating sentences from context-free and probabilistic context-free grammars, and qualitative language modeling results. A conditional version is also described that can generate sequences conditioned on sentence characteristics.

Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives [Review Article]

“…Correspondingly, adversarial training has also been investigated for both text generation [86], [87] and speech synthesis [78]. In particular, sentence generation conditioned on sentiment (either positive or negative) has been conducted in [14] and [21], but both only on fixed-length sequences (11 words in [14] and 40 words in [21]). One example can be the three generated samples with a fixed length (40 words) found in [21] (also shown in Table I).…”

Adversarial Learning of Semantic Relevance in Text to Image Synthesis

“…Curriculum learning has been successfully applied to GAN in several domains. Subramanian et al (2017) and Press et al (2017) use curriculum learning for text generation from gradually increasing the length of character sequences in text as the training progresses. Karras et al (2018) apply curriculum learning on image generation by increasing the image resolution.…”

“…Computations for probabilistic inference are often intractable, and GAN provides a viable alternative that uses neural architectures to train a generative model. GAN has been received well and applied to a variety of synthesis tasks (Karras et al 2018;Subramanian et al 2017;Radford, Metz, and Chintala 2015;Reed et al 2016b).…”

Curriculum Learning Strategies for IR

“…More recently, Bengio et al [1] provided additional evidence that curriculum strategies can benefit neural network training with experimental results on different tasks such as shape recognition and language modelling. Since then, empirical successes were observed for several computer vision [14,49] and natural language processing (NLP) tasks [36,42,60].…”