Disentangled representation learning GANs for generalized and stable font fusion network

Abstract

Automatic generation of calligraphy fonts has attracted broad attention of researchers. However, previous font generation research mainly focused on the known font style imitation based on image to image translation. For poor interpretability, it is hard for deep learning to create new fonts with various font styles and features according to human understanding. To address this issue, the font fusion network based on generative adversarial networks (GANs) and disentangled representation learning is proposed in this paper to generate brand new fonts. It separates font into two understandable disentangled features: stroke style and skeleton shape. According to personal preferences, various new fonts with multiple styles can be generated by fusing the stroke style and skeleton shape of different fonts. First, this task improves the interpretability of deep learning, and is more challenging than simply imitating font styles. Second, considering the robustness of the network, a fuzzy supervised learning skill is proposed to enhance the stability of the fusion of two fonts with considerable discrepancy. Finally, instead of retraining, the authors' trained model can be quickly transferred to other font fusion samples. It improves the efficiency of the model. Qualitative and quantitative results demonstrate that the proposed method is capable of efficiently and stably generating the new font images with multiple styles. The source code and the implementation details of our model are available at https://github.com/Qinmengxi/Fontfusion.

Generating Structurally Complete Stylish Chinese Font Based on Semi-Supervised Model

“…With the advancement of deep learning, deep neural networks have made significant progress in extracting valuable feature information from Chinese characters, encompassing strokes, radicals, and structures. These features can be efficiently integrated into GAN models, offering vital supervisory information [23][24][25][26]. However, collecting paired training data can be a time-and effort-consuming process.…”

StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding

“…As certain a type of artificial images, the components of Chinese characters such as strokes, radicals and skeletons are closely related to the font styles and structures of Chinese characters. Thus, incorporating these components into the generation of Chinese fonts has attracted an amount of attention in the past decade [10], [14], [21], [22], [23], [24], [25], [26], [27]. In the early stage, the Chinese font generation models are mainly based on the handcrafted explicit features such as strokes and radicals [21], [22], [23].…”

“…With the development of deep learning, some components of Chinese characters such as strokes, radicals and skeletons have been usually extracted by some deep neural networks and incorporated into the GAN model as certain important supervision information [10], [14], [24], [25], [26], [27]. In [24], the authors first divided Chinese characters into strokes by adopting certain a coherent point drift algorithm and then generated new font strokes by fusing the styles of two existing font strokes and further yielded new fonts by assembling them.…”

“…In [26], the authors proposed a three-stage GAN model including the skeleton extraction, skeleton transformation and stroke rendering for multi-chirography image translation. In [27], the authors proposed a font fusion network based on GAN and disentangled representation learning to generate brand new fonts, where the disentangled representation learning was used to obtain the stoke style and skeleton shape.…”