Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN

Abstract

Figure 1: Examples of combination results on sketch images (top-left) and style images (bottom-left). Our approach automatically applies the semantic features of an existing painting to an unfinished sketch. Our network has learned to classify the hair, eyes, skin and clothes, and has the ability to paint these features according to a sketch. More results can be seen at the end of paper. AbstractRecently, with the revolutionary neural style transferring methods [1,3,4,8,17], creditable paintings can be synthesized automatically from content images and style images. However, when it comes to the task of applying a painting's style to a anime sketch, these methods will just randomly colorize sketch lines as outputs ( fig. 7) and fail in the main task: specific style tranfer. In this paper, we integrated residual U-net to apply the style to the grayscale sketch with auxiliary classifier generative adversarial network (AC-GAN) [12]. The whole process is automatic and fast, and the results are creditable in the quality of art style as well as colorization.

Adversarial Colorization of Icons Based on Contour and Color Conditions

“…In addition, we compared our dual conditional GAN to state-of-the-art techniques, including iGAN [44], CycleGAN [45], conditional image-to-image translation [40], ComiColorization [8], MUNIT [? ] and Anime [42]. Experiment results demonstrate the effectiveness of our technique.…”

“…Hensman et al's method [14] required only one single reference image to train the conditional GAN [17], and used the network to colorize monochrome images. Zhang et al [42] fused the high level features extracted from sketch and style images to generate the color version of a sketch. They also applied two guided decoders to prevent the residual U-net from skipping high level features.…”

“…Both of them attempt to colorize contour images. Therefore, we compared our system to a famous online software called style2paints 2 , which was an improved version of [42]. However, because the details of this online software was not disclosed and the network was trained on Manga images, the comparison may not be fair.…”

“…However, because the details of this online software was not disclosed and the network was trained on Manga images, the comparison may not be fair. We further implemented the original network [42] and trained the network on our collected icon dataset. Figure 8 shows the results.…”

“…Specifically, a network takes a contour image drawn by the designers as input and then outputs the colorized icon image. Similar ideas have been adopted to colorize black-and-white Manga characters [5,8,14,42] and achieved great success. To control the colorization process, additional inputs, such as stroke colors and style images, are fed into the network as well.…”