Controllable Artistic Text Style Transfer via Shape-Matching GAN

Yang, Shuai; Wang, Zhangyang; Wang, Zhaowen; Xu, Ning; Liu, Jiaying; Guo, Zhongwen

doi:10.1109/iccv.2019.00454

Cited by 109 publications

(83 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A GAN has a generator network and a discriminator network playing a min-max two-player game against each other. It has achieved great success in many generation and synthesis tasks, such as text-to-image translation [66,65,61,48], image-to-image translation [22,70,63], and image enhancement [33,31,23]. However, the training of GAN is often found to be highly unstable [50], and commonly suffers from non-convergence, mode collapse, and sensitivity to hyperparameters.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

AutoGAN: Neural Architecture Search for Generative Adversarial Networks

Gong

Chang

Jiang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

278

196

View full text Add to dashboard Cite

Neural architecture search (NAS) has witnessed prevailing success in image classification and (very recently) segmentation tasks. In this paper, we present the first preliminary study on introducing the NAS algorithm to generative adversarial networks (GANs), dubbed AutoGAN. The marriage of NAS and GANs faces its unique challenges. We define the search space for the generator architectural variations and use an RNN controller to guide the search, with parameter sharing and dynamic-resetting to accelerate the process. Inception score is adopted as the reward, and a multi-level search strategy is introduced to perform NAS in a progressive way. Experiments validate the effectiveness of AutoGAN on the task of unconditional image generation. Specifically, our discovered architectures achieve highly competitive performance compared to current stateof-the-art hand-crafted GANs, e.g., setting new state-of-theart FID scores of 12.42 on CIFAR-10, and 31.01 on STL-10, respectively. We also conclude with a discussion of the current limitations and future potential of AutoGAN. The code is avaliable at https://github.com/TAMU-VITA/ AutoGAN .• We use Inception score (IS) [50] as the reward, in the reinforcement-learning-based optimization of Au-toGAN. The discovered models are found also to show

show abstract

Section: Generative Adversarial Networkmentioning

confidence: 99%

AutoGAN: Neural Architecture Search for Generative Adversarial Networks

Gong

Chang

Jiang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

278

196

View full text Add to dashboard Cite

show abstract

“…Style transfer of Gatys et al. [GEB16] and SM‐GAN [YWW*19] only partially transfer the style characteristics. SinGAN [SDM19] performs most closely to us, but the generated texture does not resemble the input texture as well.…”

Section: Resultsmentioning

confidence: 99%

“…When applied at a very fine scale, our work compares with such methods, using a completely different technique. A specific work in the context of text stylization by example [YWW*19] maps a texture to a binary map of a letter or word. This is a much more limiting scenario than ours.…”

Section: Related Workmentioning

confidence: 99%

Structural Analogy from a Single Image Pair

Benaim

Mokady

Bermano

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

The task of unsupervised image‐to‐image translation has seen substantial advancements in recent years through the use of deep neural networks. Typically, the proposed solutions learn the characterizing distribution of two large, unpaired collections of images, and are able to alter the appearance of a given image, while keeping its geometry intact. In this paper, we explore the capabilities of neural networks to understand image structure given only a single pair of images, A and B. We seek to generate images that are structurally aligned: that is, to generate an image that keeps the appearance and style of B, but has a structural arrangement that corresponds to A. The key idea is to map between image patches at different scales. This enables controlling the granularity at which analogies are produced, which determines the conceptual distinction between style and content. In addition to structural alignment, our method can be used to generate high quality imagery in other conditional generation tasks utilizing images A and B only: guided image synthesis, style and texture transfer, text translation as well as video translation. Our code and additional results are available in https://github.com/rmokady/structural-analogy/

show abstract

“…In recent years, deep learning has been widely used in many fields such as medical imaging [1], remote sensing [2], and three-dimensional modeling [3] and has played an important role in promoting the application of artificial intelligence in multiple industries. In order to discover useful macro information in the data, the purpose of deep learning is to combine low-level features to form more abstract features with strong representation ability.…”

Section: Introductionmentioning

confidence: 99%

Design of Painting Art Style Rendering System Based on Convolutional Neural Network

Xie

2021

Scientific Programming

View full text Add to dashboard Cite

Convolutional Neural Network- (CNN-) based GAN models mainly suffer from problems such as data set limitation and rendering efficiency in the segmentation and rendering of painting art. In order to solve these problems, this paper uses the improved cycle generative adversarial network (CycleGAN) to render the current image style. This method replaces the deep residual network (ResNet) of the original network generator with a dense connected convolutional network (DenseNet) and uses the perceptual loss function for adversarial training. The painting art style rendering system built in this paper is based on perceptual adversarial network (PAN) for the improved CycleGAN that suppresses the limitation of the network model on paired samples. The proposed method also improves the quality of the image generated by the artistic style of painting and further improves the stability and speeds up the network convergence speed. Experiments were conducted on the painting art style rendering system based on the proposed model. Experimental results have shown that the image style rendering method based on the perceptual adversarial error to improve the CycleGAN + PAN model can achieve better results. The PSNR value of the generated image is increased by 6.27% on average, and the SSIM values are all increased by about 10%. Therefore, the improved CycleGAN + PAN image painting art style rendering method produces better painting art style images, which has strong application value.

show abstract

Controllable Artistic Text Style Transfer via Shape-Matching GAN

Cited by 109 publications

References 29 publications

AutoGAN: Neural Architecture Search for Generative Adversarial Networks

AutoGAN: Neural Architecture Search for Generative Adversarial Networks

Structural Analogy from a Single Image Pair

Design of Painting Art Style Rendering System Based on Convolutional Neural Network

Contact Info

Product

Resources

About