AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

Xu, Tao; Zhang, Pengchuan; Huang, Qiuyuan; Zhang, Han; Gan, Zhe; Huang, Xiaolei; He, Xiaodong

doi:10.1109/cvpr.2018.00143

Cited by 1,528 publications

(1,678 citation statements)

References 27 publications

Supporting

Mentioning

1,674

Contrasting

Unclassified

Order By: Relevance

“…The generative network is used to synthesize images conditioned on the speech embedding feature. Following the recent works about text-to-images [3]- [5], we use a stacked conditional GAN, also known as StackGAN v2 [4], to synthesize the images due to its promising performance on generating photo-realistic images. As illustrated in Fig.…”

Section: Generative Networkmentioning

confidence: 99%

See 1 more Smart Citation

Direct Speech-to-Image Translation

Zhang

Jia

et al. 2020

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Direct speech-to-image translation without text is an interesting and useful topic due to the potential applications in human-computer interaction, art creation, computer-aided design. etc. Not to mention that many languages have no writing form. However, as far as we know, it has not been well-studied how to translate the speech signals into images directly and how well they can be translated. In this paper, we attempt to translate the speech signals into the image signals without the transcription stage. Specifically, a speech encoder is designed to represent the input speech signals as an embedding feature, and it is trained with a pretrained image encoder using teacher-student learning to obtain better generalization ability on new classes. Subsequently, a stacked generative adversarial network is used to synthesize high-quality images conditioned on the embedding feature. Experimental results on both synthesized and real data show that our proposed method is effective to translate the raw speech signals into images without the middle text representation. Ablation study gives more insights about our method.

show abstract

Section: Generative Networkmentioning

confidence: 99%

“…Note: the text is shown only for readability, it is not used in the speechto-image model. text-to-image translation, have been investigated in recent literature [3]- [5]. Besides, many languages have no writing form, which calls for the approaches to understand and visualize the speech directly [6].…”

Section: Introductionmentioning

confidence: 99%

Direct Speech-to-Image Translation

Zhang

Jia

et al. 2020

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

“…such as conditional variational auto-encoder (CVAE) [10]- [12] and generative adversarial network (GAN) [13]- [18]. For instance, Yan et al [11] proposed a disentangled CVAEbased method for attribute-conditioned image generation.…”

Section: Introductionmentioning

confidence: 99%

Facial Synthesis From Visual Attributes via Sketch Using Multiscale Generators

Patel

2020

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

Automatic synthesis of faces from visual attributes is an important problem in computer vision and has wide applications in law enforcement and entertainment. With the advent of deep generative convolutional neural networks (CNNs), attempts have been made to synthesize face images from attributes and text descriptions. In this paper, we take a different approach, where we formulate the original problem as a stagewise learning problem. We first synthesize the facial sketch corresponding to the visual attributes and then we generate the face image based on the synthesized sketch. The proposed framework, is based on a combination of two different Generative Adversarial Networks (GANs) -(1) a sketch generator network which synthesizes realistic sketch from the input attributes, and (2) a face generator network which synthesizes facial images from the synthesized sketch images with the help of facial attributes. Extensive experiments and comparison with recent methods are performed to verify the effectiveness of the proposed attributebased two-stage face synthesis method.

show abstract

“…Artificial Intelligence is promising in the field [11], [12] because it has been successfully applied in different fields like business for identifying job-hopping patterns [13], image recognition, etc. It can accurately and efficiently predict thousands of possible structures.…”

Section: Introductionmentioning

confidence: 99%

Artificial Intelligence for Bioinformatics: Applications in Protein Folding Prediction

Staples

Chan

et al. 2019

Preprint

View full text Add to dashboard Cite

AI recently shows great promise in the field of bioinformatics, such as protein structure prediction. The Critical Assessment of protein Structure Prediction (CASP) is a nationwide experiment that takes place biannually, which centered around analyzing the best current systems for predicting protein tertiary structures. In this paper, we research on available AI methods and features, and then explore novel methods based on reinforcement learning. Such method will have profound implications for R&D in bioinformatics and add an additional platform to the management of innovation in biotechnology.

show abstract

AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

Cited by 1,528 publications

References 27 publications

Direct Speech-to-Image Translation

Direct Speech-to-Image Translation

Facial Synthesis From Visual Attributes via Sketch Using Multiscale Generators

Artificial Intelligence for Bioinformatics: Applications in Protein Folding Prediction

Contact Info

Product

Resources

About