A Generative Adversarial Network for Multistyle Unsupervised Image Transfer Based on Dynamic Position Normalization

Abstract

Image transfer is a technology that changes the image effect by processing the image's color, contour, line, and other information. To stylize the visual appearance of the output will be adapted to the subject of the original image, this paper proposes an image feature transfer method called Cycle-DPN-GAN, based on the Cycle-Consistent Adversarial Networks. Firstly, Positional Normalization-Dynamic Moment Shortcut (PONO-DMS) module is introduced to learn more structural information from the input image, and the edge blurring and object losing are efficiently alleviated. In addition, the Multi-Scale-Structural Similarity Index (MS-SSIM) loss is added to the reconstruction loss, which improves visual perceptions and enhances the constraints on the reconstructed image in terms of image brightness, color contrast and structure. In this model, to verify the feasibility and superiority of the proposed method, the data sets of monet2photo, vangogh2photo, ukiyoe2photo and cezanne2photo are performed in the experiments, and the Inception Score and Fréchet Inception Distance evaluation index are improved. In addition, ablation studies are performed to demonstrate the validity of each proposed component. In this paper, the results of the quantitative evaluation are consistent with the qualitative evaluation. It can be demonstrated that the images generated by Cycle-DPN-GAN have higher visual quality. achieves better performance in both specific quantitative and 49 qualitative evaluation indexes. In the comparation with other 50 networks (Fig. 1), our proposed network can preserve more 51 details of the original image when obtaining a much clearer 52 stylized image. The contributions of this paper are as follows: 53 1) The structural information of the input image is 54 extracted by PONO [6] before features transforma-55 tion and then transmitted into the DMS layer, which 56 are used to recombine the stylized image. By this 57 way, the edge blurring and object losing are efficiently 58 alleviated.59 2) In order to improve the brightness, color contrast, 60 and composition, cycle consistency loss and MS-SSIM 61 loss [7] are used jointly in the generative network to 62 strengthen the constraints for realistic and natural artist 63 painting. 64 The following of this paper is organized as: Section II 65 introduces the researches of image-to-image style trans-66 fer; Section III expounds the proposed Cycle-DPN-GAN; 67 Section IV states the details of implementation, shows and 68 analyzes the experimental results; Section V gives a brief 69 conclusion of this work. 70 II. RELATED WORK 71 In this section, the recent researches of image style transfer 72 are reviewed briefly, and the related style transformation 73 methods are divided into three categories: Texture synthesis, 74 CNN based approaches and GAN based approaches.