GAN-Based Multi-Style Photo Cartoonization

Abstract

Cartoon is a common form of art in our daily life and automatic generation of cartoon images from photos is highly desirable. However, state-of-the-art single-style methods can only generate one style of cartoon images from photos and existing multi-style image style transfer methods still struggle to produce high-quality cartoon images due to their highly simplified and abstract nature. In this paper, we propose a novel multi-style generative adversarial network (GAN) architecture, called MS-CartoonGAN, which can transform photos into multiple cartoon styles. MS-CartoonGAN uses only unpaired photos and cartoon images of multiple styles for training. To achieve this, we propose to use (1) a hierarchical semantic loss with sparse regularization to retain semantic content and recover flat shading in different abstract levels, (2) a new edge-promoting adversarial loss for producing fine edges, and (3) a style loss to enhance the difference between output cartoon styles and make training process more stable. We also develop a multi-domain architecture, where the generator consists of a shared encoder and multiple decoders for different cartoon styles, along with multiple discriminators for individual styles. By observing that cartoon images drawn by different artists have their unique styles while sharing some common characteristics, our shared network architecture exploits the common characteristics of cartoon styles, achieving better cartoonization and being more efficient than single-style cartoonization. We show that our multi-domain architecture can theoretically guarantee to output desired multiple cartoon styles. Through extensive experiments including a user study, we demonstrate the superiority of the proposed method, outperforming state-of-the-art single-style and multi-style image style transfer methods.

Multi-style Cartoon Style Migration Study

“…If the output image does not clearly reproduce the edges, but has the correct shading, it becomes difficult to train the discriminator. To circumvent this problem, in this paper, we use guided filtering for edge smoothing on the trained cartoon image [16], where general filters are isotropic filters, such as Gaussian filters, which smooth the image while erasing some of the edge frame details. Whereas guided filtering smooths the graph without blurring out the image edges and contours.…”

Evaluation Metrics for Intelligent Generation of Graphical Game Assets: A Systematic Survey-Based Framework

“…FID utilises Frechet distance between the two image distributions, and assumes that the two are Gaussian. This is applied to 2D assets [7], [64], [105], [106], [107], [108], [109], 3D assets via 3D classifiers [37], [110] or rasterisation to 2D form [111], [112].The Frechet point cloud distance extends FID for applications in assessing the similarity of pointbased 3D shapes [113]. This has been used to evaluate many deep-learning based 3D point-cloud [71], [113], [114] and mesh [67] generators.…”

Learning Gaze-aware Compositional GAN from Limited Annotations

“…Synthetic data generation and augmentation using generative models might be a promising alternative. Generative adversarial networks (GAN) can produce high-quality images with a high level of realism [Karras et al 2019] with the added benefit of controlling the generated data by editing latent input vectors or conditioning the model on some input information [Shu et al 2021].…”