A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis

Agnese, Jorge; Herrera, Jonathan; Tao, Haicheng; Zhu, Xingquan

doi:10.48550/arxiv.1910.09399

Cited by 5 publications

(5 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the original GANs frameworks were initially built upon images, there is no doubt that the number of GANs applications in the image domain surpasses other areas such as text, voice and video. Multiple reviews [28][29][30][31][32] focus on image synthesis, even though the major proportion of the general GANs reviews mentioned above, are also in the image domain. Huang et al [29] categorize the image synthesis GANs frameworks into three types based on the overall architecture.…”

Section: Image Gansmentioning

confidence: 99%

“…Similar applications are also reviewed elsewhere [28], with additional applications such as face ageing and 3D image synthesis. Moreover, Agnese et al [32] direct attention to GANs models that are conditioned on text and produce images. Some of these models fall under semantic enhancement GANs, where the main goal is to ensure that text is semantically coherent with the generated image.…”

Section: Image Gansmentioning

confidence: 99%

See 1 more Smart Citation

Video Generative Adversarial Networks: A Review

et al. 2022

View full text Add to dashboard Cite

With the increasing interest in the content creation field in multiple sectors such as media, education, and entertainment, there is an increased trend in the papers that use AI algorithms to generate content such as images, videos, audio, and text. Generative Adversarial Networks (GANs) is one of the promising models that synthesizes data samples that are similar to real data samples. While the variations of GANs models in general have been covered to some extent in several survey papers, to the best of our knowledge, this is the first paper that reviews the state-of-the-art video GANs models. This paper first categorizes GANs review papers into general GANs review papers, image GANs review papers, and special field GANs review papers such as anomaly detection, medical imaging, or cybersecurity. The paper then summarizes the main improvements in GANs that are not necessarily applied in the video domain in the first run but have been adopted in multiple video GANs variations. Then, a comprehensive review of video GANs models are provided under two main divisions based on existence of a condition. The conditional models are then further classified according to the provided condition into audio, text, video, and image. The paper concludes with the main challenges and limitations of the current video GANs models.

show abstract

Section: Image Gansmentioning

confidence: 99%

Section: Image Gansmentioning

confidence: 99%

Video Generative Adversarial Networks: A Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…We focus on methods developed for Generative Adversarial Networks (GANs) (Goodfellow et al, 2014) that have demonstrated the most impressive results to date. The interested reader can find dedicated surveys on each conditional task (Agnese et al, 2019;Wu et al, 2017). The literature on conditional generative models (Mirza & Osindero, 2014) can be clustered depending on the type of conditional data.…”

Section: Conditional Generative Modelsmentioning

confidence: 99%

CoPE: Conditional image generation using Polynomial Expansions

Chrysos¹,

Georgopoulos²,

Panagakis³

2021

Preprint

View full text Add to dashboard Cite

Generative modeling has evolved to a notable field of machine learning. Deep polynomial neural networks (PNNs) have demonstrated impressive results in unsupervised image generation, where the task is to map an input vector (i.e., noise) to a synthesized image. However, the success of PNNs has not been replicated in conditional generation tasks, such as super-resolution. Existing PNNs focus on single-variable polynomial expansions which do not fare well to two-variable inputs, i.e., the noise variable and the conditional variable. In this work, we introduce a general framework, called CoPE, that enables a polynomial expansion of two input variables and captures their auto-and cross-correlations. We exhibit how CoPE can be trivially augmented to accept an arbitrary number of input variables. CoPE is evaluated in five tasks (class-conditional generation, inverse problems, edges-to-image translation, image-to-image translation, attribute-guided generation) involving eight datasets. The thorough evaluation suggests that CoPE can be useful for tackling diverse conditional generation tasks.

show abstract

“…T2I has made significant progress in the last 5 years. Thus, several studies [8][9][10][11] have put forth a semantic taxonomy for adversarial text-to-image synthesis (T2I), summarizing the efforts made using GAN [12] mainly. In contrast, this paper focuses on two primary gaps in previous studies.…”

Section: Introductionmentioning

confidence: 99%

“…Viewing T2I as a vast domain, we comprehensively present a semantic taxonomy of Text-guided Visual-Output (T2Vo) contrary to either text-to-image synthesis [8] or T2I using GAN exclusively [9][10][11].…”

mentioning

confidence: 99%

A Review of Multi-Modal Learning from the Text-Guided Visual Processing Viewpoint

Ullah

Lee

et al. 2022

Sensors

View full text Add to dashboard Cite

For decades, co-relating different data domains to attain the maximum potential of machines has driven research, especially in neural networks. Similarly, text and visual data (images and videos) are two distinct data domains with extensive research in the past. Recently, using natural language to process 2D or 3D images and videos with the immense power of neural nets has witnessed a promising future. Despite the diverse range of remarkable work in this field, notably in the past few years, rapid improvements have also solved future challenges for researchers. Moreover, the connection between these two domains is mainly subjected to GAN, thus limiting the horizons of this field. This review analyzes Text-to-Image (T2I) synthesis as a broader picture, Text-guided Visual-output (T2Vo), with the primary goal being to highlight the gaps by proposing a more comprehensive taxonomy. We broadly categorize text-guided visual output into three main divisions and meaningful subdivisions by critically examining an extensive body of literature from top-tier computer vision venues and closely related fields, such as machine learning and human–computer interaction, aiming at state-of-the-art models with a comparative analysis. This study successively follows previous surveys on T2I, adding value by analogously evaluating the diverse range of existing methods, including different generative models, several types of visual output, critical examination of various approaches, and highlighting the shortcomings, suggesting the future direction of research.

show abstract

A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis

Cited by 5 publications

References 51 publications

Video Generative Adversarial Networks: A Review

Video Generative Adversarial Networks: A Review

CoPE: Conditional image generation using Polynomial Expansions

A Review of Multi-Modal Learning from the Text-Guided Visual Processing Viewpoint

Contact Info

Product

Resources

About