Styled and characteristic Peking opera facial makeup synthesis with Co-training and Transfer Conditional StyleGAN2

Abstract

Against the backdrop of the deep integration of culture and technology, research and practice in digitization of intangible cultural heritage has continued to deepen. However, due to the lack of data and training, it is still very difficult to apply artificial intelligence to the field of cultural heritage protection. This article integrates image generation technology into the digital protection of Peking opera facial makeup, using a self-built Peking opera facial makeup dataset. Based on the StyleGAN2 network, we propose a style generative cooperative training network Co-StyleGAN2, which integrates the Adaptive Data Augmentation to alleviate the problem of discriminator overfitting and introduces the idea of cooperative training to design a dual discriminator collaborative training network structure to stabilize the training process. We designed a Peking opera facial makeup image conditional generation network TC-StyleGAN2 which is transferred from unconditional generation network. The weights of the unconditional pre-training model are fixed, and an adaptive filtering modulation module is added to modulate the category parameters to complete the conversion from unconditional to conditional StyleGAN2 to deal with the training difficulty of conditional GANs on limited data, which suffer from severe mode collapse. The experimental results shows that the training strategy proposed in this article is better than the comparison algorithm, and the image generation quality and diversity have been improved.