Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis

Mao, Qi; Lee, Hsin-Ying; Tseng, Hung-Yu; Ma, Siwei; Yang, Ming–Hsuan

doi:10.1109/cvpr.2019.00152

Cited by 379 publications

(326 citation statements)

References 16 publications

(65 reference statements)

Supporting

Mentioning

325

Contrasting

Order By: Relevance

“…For BicycleGAN [Zhu et al 2017b], although it can capture the content information and output roughly correct glyph shapes for the English dataset, it can not process the style information well. The more recent work, MS-Pix2Pix [Mao et al 2019], can not even handle the content information well, and sometimes outputs glyph images with incorrect contents. What is worse, both of these two methods are prone to mode collapse, and can not even converge on the Chinese dataset which is more challenging.…”

Section: Pre-training Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Artistic glyph image synthesis via one-stage few-shot learning

et al. 2019

View full text Add to dashboard Cite

Fig. 1. (a) An overview of our method, given a few reference samples (5 for English or 30 for Chinese), glyph images of all other characters in the same style can be synthesized. (b) Examples of synthesized English/Chinese glyph images obtained by our proposed AGIS-Net, MC-GAN [Azadi et al. 2018] and TET-GAN [Yang et al. 2019], respectively, please zoom in for better inspection.Automatic generation of artistic glyph images is a challenging task that attracts many research interests. Previous methods either are specifically designed for shape synthesis or focus on texture transfer. In this paper, we propose a novel model, AGIS-Net, to transfer both shape and texture styles in one-stage with only a few stylized samples. To achieve this goal, we first disentangle the representations for content and style by using two encoders, ensuring the multi-content and multi-style generation. Then we utilize two collaboratively working decoders to generate the glyph shape image and its texture image simultaneously. In addition, we introduce a local texture refinement loss to further improve the quality of the synthesized textures. In this manner, our one-stage model is much more efficient and effective than other multi-stage stacked methods. We also propose a large-scale dataset with Chinese glyph images in various shape and texture styles, rendered from 35 professional-designed artistic fonts with 7,326 characters and 2,460 synthetic artistic fonts with 639 characters, to validate the effectiveness and extendability of our method. Extensive experiments on both English and Chinese artistic glyph image datasets demonstrate the superiority of our model in generating high-quality stylized glyph images against other state-of-the-art methods.

show abstract

Section: Pre-training Resultsmentioning

confidence: 99%

“…Examples of some glyph images generated by BicycleGAN[Zhu et al 2017b], MS-Pix2Pix[Mao et al 2019] and our AGIS-Net, on English and Chinese glyph image datasets during pre-training.…”

mentioning

confidence: 99%

Artistic glyph image synthesis via one-stage few-shot learning

et al. 2019

View full text Add to dashboard Cite

show abstract

“…By varying interpolation coefficients a, we expect to generate diverse images, but one common problem for GAN is mode collapse [27], which means that the generated images may collapse into a few modes. In our fusion generator, when sampling two different interpolation coefficients a 1 and a 2 , the generated images G(a 1 , X S ) and G(a 2 , X S ) are likely to collapse into the same mode.…”

Section: Fusion Discriminatormentioning

confidence: 99%

“…To ensure the diversity of generated images, we additionally employ a mode seeking loss and an interpolation regression loss, both of which are related to interpolation coefficients. Specifically, we use a variant of mode seeking loss [27] to prevent the images generated based on different interpolation coefficients from collapsing to a few modes. Moreover, we propose a novel interpolation regression loss by regressing the interpolation coefficients based on the features of conditional images and generated image, which means that each generated image can recognize its corresponding interpolation coefficients.…”

Section: Introductionmentioning

confidence: 99%

F2GAN: Fusing-and-Filling GAN for Few-shot Image Generation

Yan

Niu

Zhang

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

In order to generate images for a given category, existing deep generative models generally rely on abundant training images. However, extensive data acquisition is expensive and fast learning ability from limited data is necessarily required in real-world applications. Also, these existing methods are not well-suited for fast adaptation to a new category. Few-shot image generation, aiming to generate images from only a few images for a new category, has attracted some research interest. In this paper, we propose a Fusing-and-Filling Generative Adversarial Network (F2GAN) to generate realistic and diverse images for a new category with only a few images. In our F2GAN, a fusion generator is designed to fuse the high-level features of conditional images with random interpolation coefficients, and then fills in attended low-level details with non-local attention module to produce a new image. Moreover, our discriminator can ensure the diversity of generated images by a mode seeking loss and an interpolation regression loss. Extensive experiments on five datasets demonstrate the effectiveness of our proposed method for few-shot image generation. CCS CONCEPTS • Computing methodologies → Image representations; Neural networks; Learning latent representations.

show abstract

“…Generative adversarial network [23], as an emerging generative algorithm, has made great progress in the quality and variety of generated images [24][25][26][27][28][29][30][31][32][33][34][35][36]. GAN can imitate the distribution of original data and output new samples with similar characteristics.…”

Section: Introductionmentioning

confidence: 99%

Res-WGAN: Image Classification for Plant Small-scale Datasets

Xia

Yang

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Artificial identification of rare plants is an important yet challenging problem in plant taxonomy. Although deep learning-based method can accurately predict rare plant category from training samples, accuracy requirements of only few experts are satisfied due to the small sample size. Thus, effective data augmentation is vital to improve the expressive power and robustness of deep learning models, especially for plant small-sample classification tasks. Different from traditional methods, a generative adversarial network (GAN) can mimic the distribution of primary data and produce similar but nonidentical samples. Data augmentation for automated classification of rare plant samples with GAN has not been studied for a long time. Result: In this study, we present a novel GAN model, referred to as residual Wasserstein GAN (Res-WGAN), for extending rare plant dataset efficiently. With a premise of minimized size of training parameters, residual block serves as elementary building block to increase the model’s expression ability. A loss function based on Wasserstein distance with gradient penalty is used to ensure diversity of output. Moreover, we draw on the super-resolution GAN (SRGAN) and consider perceptual loss into the function. Perceptual loss guarantees similarity between generated samples and original samples in high-dimensional features. Benefiting from these improvements, the expanded dataset improves the classification accuracy on the residual neural network (ResNet), a classic convolutional neural network, and restrains the overfitting phenomenon effectively by using transfer learning. More similar experiments are presented in this paper to prove the practicality of the model. Conclusions: Our proposed method produces better diversity and sharpness of generated samples and more accurate classification results than other data augmentation methods. In addition, a user study confirms that it is an ideal alternative strategy for small-scale rare plant identification. Developing a robust and effective small-scale plant classification method to replace expert testimony is highly relevant for agricultural automation development.

show abstract

Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis

Cited by 379 publications

References 16 publications

Artistic glyph image synthesis via one-stage few-shot learning

Artistic glyph image synthesis via one-stage few-shot learning

F2GAN: Fusing-and-Filling GAN for Few-shot Image Generation

Res-WGAN: Image Classification for Plant Small-scale Datasets

Contact Info

Product

Resources

About