No abstract

No abstract

No abstract

In this paper, we investigate the underlying factor that leads to failure and success in the training of GANs. We study the property of the optimal discriminative function and show that in many GANs, the gradient from the optimal discriminative function is not reliable, which turns out to be the fundamental cause of failure in the training of GANs. We further demonstrate that a well-defined distance metric does not necessarily guarantee the convergence of GANs. Finally, we prove in this paper that Lipschitz-continuity condition is a general solution to make the gradient of the optimal discriminative function reliable, and characterized the necessary condition where Lipschitz-continuity ensures the convergence, which leads to a broad family of valid GAN objectives under Lipschitz-continuity condition, where Wasserstein distance is one special case. We experiment with several new objectives, which are sound according to our theorems, and we found that, compared with Wasserstein distance, the outputs of the discriminator with new objectives are more stable and the final qualities of generated samples are also consistently higher than those produced by Wasserstein distance.

Typical methods for text-to-image synthesis seek to design effective generative architecture to model the text-toimage mapping directly. It is fairly arduous due to the cross-modality translation involved in the task of text-toimage synthesis. In this paper we circumvent this problem by focusing on parsing the content of both the input text and the synthesized image thoroughly to model the text-to-image consistency in the semantic level. In particular, we design a memory structure to parse the textual content by exploring semantic correspondence between each word in the vocabulary to its various visual contexts across relevant images in training data during text encoding. On the other hand, the synthesized image is parsed to learn its semantics in an object-aware manner. Moreover, we customize a conditional discriminator, which models the fine-grained correlations between words and image sub-regions to push for the cross-modality semantic alignment between the input text and the synthesized image. Thus, a full-spectrum contentoriented parsing in the deep semantic level is performed by our model, which is referred to as Content-Parsing Generative Adversarial Networks (CPGAN). Extensive experiments on COCO dataset manifest that CPGAN advances the state-of-the-art performance significantly.