Crossing-Domain Generative Adversarial Networks for Unsupervised Multi-Domain Image-to-Image Translation

Yang, Xuewen; Xie, Dong; Wang, Xin

doi:10.1145/3240508.3240716

Cited by 35 publications

(17 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CD-GAN [33], RG-UNIT [34] and MUNIT [5] are proposed for multi-modal unsupervised image-to-image translation. They all use a latent-code-sharing assumption.…”

Section: Multi-domain Image-to-image Synthesismentioning

confidence: 99%

See 1 more Smart Citation

GAN-Based Multi-Style Photo Cartoonization

Shu

Xia

et al. 2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Cartoon is a common form of art in our daily life and automatic generation of cartoon images from photos is highly desirable. However, state-of-the-art single-style methods can only generate one style of cartoon images from photos and existing multi-style image style transfer methods still struggle to produce high-quality cartoon images due to their highly simplified and abstract nature. In this paper, we propose a novel multi-style generative adversarial network (GAN) architecture, called MS-CartoonGAN, which can transform photos into multiple cartoon styles. MS-CartoonGAN uses only unpaired photos and cartoon images of multiple styles for training. To achieve this, we propose to use (1) a hierarchical semantic loss with sparse regularization to retain semantic content and recover flat shading in different abstract levels, (2) a new edge-promoting adversarial loss for producing fine edges, and (3) a style loss to enhance the difference between output cartoon styles and make training process more stable. We also develop a multi-domain architecture, where the generator consists of a shared encoder and multiple decoders for different cartoon styles, along with multiple discriminators for individual styles. By observing that cartoon images drawn by different artists have their unique styles while sharing some common characteristics, our shared network architecture exploits the common characteristics of cartoon styles, achieving better cartoonization and being more efficient than single-style cartoonization. We show that our multi-domain architecture can theoretically guarantee to output desired multiple cartoon styles. Through extensive experiments including a user study, we demonstrate the superiority of the proposed method, outperforming state-of-the-art single-style and multi-style image style transfer methods.

show abstract

“…CD-GAN [33], RG-UNIT [34] and MUNIT [5] are proposed for multi-modal unsupervised image-to-image translation. They all use a latent-code-sharing assumption.…”

Section: Multi-domain Image-to-image Synthesismentioning

confidence: 99%

“…They all use a latent-code-sharing assumption. CD-GAN [33] learns high-level features across different domains and can generate diverse and realistic results. RG-UNIT [34] takes the power of a retrieval system to complete translation.…”

Section: Multi-domain Image-to-image Synthesismentioning

confidence: 99%

GAN-Based Multi-Style Photo Cartoonization

Shu

Xia

et al. 2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…A different line of work is multi-domain image-to-image translation [5,2,52]: here, the same model can be used for translating images according to multiple attributes (i.e., hair color, gender or age). Other methods, instead, focus on diverse image-to-image translation, in which an image can be translated in multiple ways by encoding different style properties of the target distribution [57,15,24].…”

Section: Related Workmentioning

confidence: 99%

Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-To-Image Translation

Tomei

Cornia

Baraldi

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

The applicability of computer vision to real paintings and artworks has been rarely investigated, even though a vast heritage would greatly benefit from techniques which can understand and process data from the artistic domain. This is partially due to the small amount of annotated artistic data, which is not even comparable to that of natural images captured by cameras. In this paper, we propose a semantic-aware architecture which can translate artworks to photo-realistic visualizations, thus reducing the gap between visual features of artistic and realistic data. Our architecture can generate natural images by retrieving and learning details from real photos through a similarity matching strategy which leverages a weakly-supervised semantic understanding of the scene. Experimental results show that the proposed technique leads to increased realism and to a reduction in domain shift, which improves the performance of pre-trained architectures for classification, detection, and segmentation. Code is publicly available at: https://github.com/aimagelab/art2real.

show abstract

“…DualGAN [24] suggested a similar method to CycleGAN but used a Wasserstein loss [25] instead of the least-square loss. UNIT [26] and CD-GAN [27] used a shared latent space and cycle consistency for better quality and accuracy.…”

Section: Gan-based Image-to-image Translationmentioning

confidence: 99%

Image-To-Image Translation Using a Cross-Domain Auto-Encoder and Decoder

2019

View full text Add to dashboard Cite

Recently, several studies have focused on image-to-image translation. However, the quality of the translation results is lacking in certain respects. We propose a new image-to-image translation method to minimize such shortcomings using an auto-encoder and an auto-decoder. This method includes pre-training two auto-encoders and decoder pairs for each source and target image domain, cross-connecting two pairs and adding a feature mapping layer. Our method is quite simple and straightforward to adopt but very effective in practice, and we experimentally demonstrated that our method can significantly enhance the quality of image-to-image translation. We used the well-known cityscapes, horse2zebra, cat2dog, maps, summer2winter, and night2day datasets. Our method shows qualitative and quantitative improvements over existing models. Appl. Sci. 2019, 9, 4780 2 of 16 translation method. In this paper, we propose a new translation method where the mapping process from source domain to target domain can work better, and thus suppress mistranslation by transferring the learning of domain and target feature spaces using both an auto-encoder and decoder. In contrast to commonly-used feature space learning methods [9][10][11][12][13], our approach utilizes an auto-encoder and an auto-decoder-which are pretrained for the source domain and target domain, respectively-and cross-connects them by mapping their feature spaces across the two domains. Appl. Sci. 2019, 9, x FOR PEER REVIEW 2 of 17 Author Contributions: Conceptualization and methodology, J.Y., H.E., and Y.S.C.; investigation, J.Y. and H.E.; data curation and conceiving experiments, J.Y.; performing experiments and designing results, H.E.; writing-original draft preparation, J.Y. and H.E.; writing-review and editing, J.Y., H.E., and Y.S.C. Appendix AWe provide our source code at https://github.com/Rooooss/I2I_CD_AE.data curation and conceiving experiments, J.Y.; performing experiments and designing results, H.E.; writingoriginal draft preparation, J.Y. and H.E.; writing-review and editing, J.Y., H.E., and Y.S.C.

show abstract

Crossing-Domain Generative Adversarial Networks for Unsupervised Multi-Domain Image-to-Image Translation

Cited by 35 publications

References 13 publications

GAN-Based Multi-Style Photo Cartoonization

GAN-Based Multi-Style Photo Cartoonization

Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-To-Image Translation

Image-To-Image Translation Using a Cross-Domain Auto-Encoder and Decoder

Contact Info

Product

Resources

About