CJE-TIG: Zero-shot cross-lingual text-to-image generation by Corpora-based Joint Encoding

Zhang, Han; Yang, Suyi; Zhu, Hongqing

doi:10.1016/j.knosys.2021.108006

Cited by 5 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, The CJE-TIG [56] cross-lingual text-toimage pre-training technique removes barriers to using GANbased text-to-image synthesis models for any given input language. This method alters text-to-image training patterns that are linguistically specific.…”

Section: A Text To Image Generation Using Gansmentioning

confidence: 99%

See 1 more Smart Citation

Text-to-Image Synthesis With Generative Models: Methods, Datasets, Performance Metrics, Challenges, and Future Direction

Alhabeeb,

Al-Shargabi

2024

IEEE Access

View full text Add to dashboard Cite

Text-to-image synthesis, the process of turning words into images, opens up a world of creative possibilities, and meets the growing need for engaging visual experiences in a world that is becoming more image-based. As machine learning capabilities expanded, the area progressed from simple tools and systems to robust deep learning models that can automatically generate realistic images from textual inputs. Modern, large-scale text-to-image generation models have made significant progress in this direction, producing diversified and high-quality images from text description prompts. Although several methods exist, Generative Adversarial Networks (GANs) have long held a position of prominence. However, diffusion models have recently emerged, with results much beyond those achieved by GANs. This study offers a concise overview of text-to-image generative models by examining the existing body of literature and provide a deeper understanding of this topic. This will be accomplished by providing a concise summary of the development of text-to-image synthesis, previous tools and systems employed in this field, key types of generative models, as well as an exploration of the relevant research conducted on GANs and diffusion models. Additionally, the study provides an overview of common datasets utilized for training the text-toimage model, compare the evaluation metrics used for evaluating the models and addresses the challenges encountered in the field. Finally, concluding remarks are provided to summarize the findings and implications of the study and open issues for further research.

show abstract

Section: A Text To Image Generation Using Gansmentioning

confidence: 99%

“…To overcome the language barrier, some studies proposed multilingual [95] and cross-lingual [56] models to support multiple languages within the same model. The goal of these multilingual models is to break down linguistic barriers by providing a common groundwork for the comprehension and processing of several languages at once.…”

Section: Future Directionsmentioning

confidence: 99%

Text-to-Image Synthesis With Generative Models: Methods, Datasets, Performance Metrics, Challenges, and Future Direction

Alhabeeb,

Al-Shargabi

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Nevertheless, the discriminator aims to distinguish between the synthetic image and the matched natural image. The input conditions used by GAN-based image synthesis methods are various, such as sparse sketches [19][20][21] , gaussian noise 22,23 , text descriptions [24][25][26] , natural images 27,28 , and semantic layout [29][30][31][32] . Considering the great success of GANs in image synthesis, we propose a novel GAN-based approach to tackle image synthesis conditioned only on semantic layout.…”

Section: Introductionmentioning

confidence: 99%

UNet-like network fused Swin Transformer and CNN for Semantic Image Synthesis

Ke,

Luo,

Cai

2024

Preprint

View full text Add to dashboard Cite

Semantic image synthesis approaches has been dominated by the modelling of Convolutional Neural Networks (CNN). Due to the limitations of local perception, their performance improvement seems to have plateaued in recent years. To tackle this issue, we propose the TransUNet model, which is a UNet-like network fused Swin Transformer and CNN for semantic image synthesis. Photorealistic image synthesis conditional on the given semantic layout depends on the high-level semantics and the low-level positions. To improve the synthesis performance, we design a novel conditional residual fusion module for the model decoder to efficiently fuse the hierarchical feature maps extracted at different scales. Moreover, this module combines the opposition-based learning mechanism and the weight assignment mechanism for enhancing and attending the semantic information. Compared to pure CNN-based models, our TransUNet combines the local and global perceptions to better extract high- and low-level features and better fuse multi-scale features. We have conducted an extensive amount of comparison experiments, both in quantitative and qualitative terms, to validate the effectiveness of our proposed TransUNet model for semantic image synthesis. The outcomes illustrate that TransUNet distinctively outperforms the state-of-the-art model on three benchmark datasets (Citysacpes, ADE20K, and COCO-Stuff) including numerous real-scene images.

show abstract

CMAFGAN: A Cross-Modal Attention Fusion based Generative Adversarial Network for attribute word-to-face synthesis

Luo¹,

Chen²,

He³

et al. 2022

Knowledge-Based Systems

View full text Add to dashboard Cite

CJE-TIG: Zero-shot cross-lingual text-to-image generation by Corpora-based Joint Encoding

Cited by 5 publications

References 25 publications

Text-to-Image Synthesis With Generative Models: Methods, Datasets, Performance Metrics, Challenges, and Future Direction

Text-to-Image Synthesis With Generative Models: Methods, Datasets, Performance Metrics, Challenges, and Future Direction

UNet-like network fused Swin Transformer and CNN for Semantic Image Synthesis

CMAFGAN: A Cross-Modal Attention Fusion based Generative Adversarial Network for attribute word-to-face synthesis

Contact Info

Product

Resources

About