Improvements on style transfer from photographs to Chinese ink wash paintings based on generative adversarial networks

Abstract

Although voids play an important role in Chinese paintings, when generating Chinese ink wash paintings from real life photos of horses, background detection and void-leaving remain challenges in producing more credible paintings. To address the problem, a model with a two-stage framework is proposed in this paper. The framework divides the generation process into background lightening and style transformation. In the first stage, by training a pix2pix model with paired original photos and background-lightened photos, the pix2pix model is enabled to detect and lighten the background correctly in the style-transfer process, hinting where voids should be left. In the second stage, a Cycle-GAN model trained with unmatched photos and Chinese paintings achieves the style-transfer from pre-processed photos to Chinese ink wash paintings. In preparation for training the stage-Ⅰ model, the photos from a given dataset is processed in batches to get corresponding background-lightened photos. During the training of the stage-Ⅱ model, original photos instead of processed photos is used as comparatively data augmentation, making the model more robust and independent. Compared to the baseline model, the proposed model reaches a higher accuracy in both void-leaving and detailenhancing in the experiments, resulting in more credible and delicate generated Chinese paintings.