Text-Guided Image Manipulation via Generative Adversarial Network With Referring Image Segmentation-Based Guidance

Watanabe, Yuto; Togo, Ren; Maeda, Kazuo; Ogawa, Takahiro; Haseyama, Miki

doi:10.1109/access.2023.3269847

Cited by 3 publications

(2 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The research on text-guided image editing was rapidly accelerated by the emergence of the generative adversarial network (GAN) [39]. Approaches to GAN-based text-guided image editing can be divided into two categories: (1) the approaches [10,11,[40][41][42][43] utilizing a unique network with a single or multi-stage architecture, and (2) the approaches [12,13,[44][45][46] leveraging the representation capabilities of a pretrained StyleGAN [16,47,48]. In approach (1), some studies [40,41] have applied an encoder-decoder architecture and successfully generated 64 × 64 resolution edited images on datasets such as Oxford-102 flower [49] and Caltech-UCSD Birds [50].…”

Section: Text-guided Image Editingmentioning

confidence: 99%

“…In approach (1), some studies [40,41] have applied an encoder-decoder architecture and successfully generated 64 × 64 resolution edited images on datasets such as Oxford-102 flower [49] and Caltech-UCSD Birds [50]. To generate high-resolution edited images on complex image dataset such as MSCOCO [51], several studies [10,11,42,43] construct a multi-stage architecture with a generator and discriminator at each stage. Three stages are trained at the same time, and progressively generate edited images of three different resolutions, i.e., 64 × 64 → 128 × 128 → 256 × 256.…”

Section: Text-guided Image Editingmentioning

confidence: 99%

See 1 more Smart Citation

Text-Guided Image Editing Based on Post Score for Gaining Attention on Social Media

Watanabe,

Togo,

Maeda

et al. 2024

Sensors

Self Cite

View full text Add to dashboard Cite

Text-guided image editing has been highlighted in the fields of computer vision and natural language processing in recent years. The approach takes an image and text prompt as input and aims to edit the image in accordance with the text prompt while preserving text-unrelated regions. The results of text-guided image editing differ depending on the way the text prompt is represented, even if it has the same meaning. It is up to the user to decide which result best matches the intended use of the edited image. This paper assumes a situation in which edited images are posted to social media and proposes a novel text-guided image editing method to help the edited images gain attention from a greater audience. In the proposed method, we apply the pre-trained text-guided image editing method and obtain multiple edited images from the multiple text prompts generated from a large language model. The proposed method leverages the novel model that predicts post scores representing engagement rates and selects one image that will gain the most attention from the audience on social media among these edited images. Subject experiments on a dataset of real Instagram posts demonstrate that the edited images of the proposed method accurately reflect the content of the text prompts and provide a positive impression to the audience on social media compared to those of previous text-guided image editing methods.

show abstract

Section: Text-guided Image Editingmentioning

confidence: 99%

Section: Text-guided Image Editingmentioning

confidence: 99%

Text-Guided Image Editing Based on Post Score for Gaining Attention on Social Media

Watanabe,

Togo,

Maeda

et al. 2024

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

Advancements in Generative Modeling: A Comprehensive Survey of GANs and Diffusion Models for Text-to-Image Synthesis and Manipulation

Deshmukh,

Ambulkar,

Sarjoshi

et al. 2024

2024 IEEE International Students' Conference on Electrical, Electronics and Computer Science (SCEECS)

View full text Add to dashboard Cite

Spectral Normalization for Generative Adversarial Networks for Artistic Image Transformation

Shu,

Zhang

2024

International Journal of Digital Multimedia Broadcasting

View full text Add to dashboard Cite

Artistic image transformation is a computer technique widely applied in art creation, design, entertainment, and cultural heritage by converting images into artistic styles. It offers innovative ways for artists to express themselves, provides designers with more choices and inspiration, enhances visual esthetics, and enables creative implementations in movies, games, and virtual reality. Additionally, it aids in the restoration and preservation of ancient artworks, allowing a deeper appreciation of classical art. Traditional image transformation methods, though effective for simple effects, lack the flexibility and expressiveness of deep learning–based approaches. To enhance the effectiveness and efficiency of artistic image transformation, this paper employs generative adversarial networks (GANs), which utilize an adversarial training mechanism between a generator and a discriminator to produce high‐quality and realistic image transformations. This study introduces spectral normalization (SNGAN) to further improve GAN performance by constraining the spectral norm of the discriminator’s weight matrix, preventing gradient issues during training, thus improving convergence and image quality. Experimental results on the CHAOS dataset indicate that the proposed SNGAN model achieves the lowest mean absolute error (MAE) of 0.3420, the highest peak signal‐to‐noise ratio (PSNR) of 32.1423, and a structural similarity index (SSIM) of 0.6696, closely matching the best result. Additionally, the SNGAN model demonstrates the shortest training time, highlighting its efficiency. These results confirm that the proposed method achieves more realistic and efficient artistic image transformations compared to traditional methods and other deep learning algorithms.

show abstract

Text-Guided Image Manipulation via Generative Adversarial Network With Referring Image Segmentation-Based Guidance

Cited by 3 publications

References 38 publications

Text-Guided Image Editing Based on Post Score for Gaining Attention on Social Media

Text-Guided Image Editing Based on Post Score for Gaining Attention on Social Media

Advancements in Generative Modeling: A Comprehensive Survey of GANs and Diffusion Models for Text-to-Image Synthesis and Manipulation

Spectral Normalization for Generative Adversarial Networks for Artistic Image Transformation

Contact Info

Product

Resources

About