SE-GAN: Skeleton Enhanced GAN-based Model for Brush Handwriting Font Generation

Abstract

Previous works on font generation mainly focus on the standard print fonts where character's shape is stable and strokes are clearly separated. There is rare research on brush handwriting font generation, which involves holistic structure changes and complex strokes transfer. To address this issue, we propose a novel GAN-based image translation model by integrating the skeleton information. We first extract the skeleton from training images, then design an image encoder and a skeleton encoder to extract corresponding features. A self-attentive refined attention module is devised to guide the model to learn distinctive features between different domains. A skeleton discriminator is involved to first synthesize the skeleton image from the generated image with a pre-trained generator, then to judge its realness to the target one. We also contribute a large-scale brush handwriting font image dataset with six styles and 15,000 high-resolution images. Both quantitative and qualitative experimental results demonstrate the competitiveness of our proposed model.

StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding

“…As certain a type of artificial images, the components of Chinese characters such as strokes, radicals and skeletons are closely related to the font styles and structures of Chinese characters. Thus, incorporating these components into the generation of Chinese fonts has attracted an amount of attention in the past decade [10], [14], [21], [22], [23], [24], [25], [26], [27]. In the early stage, the Chinese font generation models are mainly based on the handcrafted explicit features such as strokes and radicals [21], [22], [23].…”

“…With the development of deep learning, some components of Chinese characters such as strokes, radicals and skeletons have been usually extracted by some deep neural networks and incorporated into the GAN model as certain important supervision information [10], [14], [24], [25], [26], [27]. In [24], the authors first divided Chinese characters into strokes by adopting certain a coherent point drift algorithm and then generated new font strokes by fusing the styles of two existing font strokes and further yielded new fonts by assembling them.…”

“…With the emergence of the generative adversarial network (GAN) [6], the kind of models based on GAN become the mainstream for the generation of Chinese fonts. These models can be generally divided into two categories, i.e., the class of models based on the paired data (that is, there is a one-to-one correspondence between the source and target font domains as shown in Figure 2(c)) [7], [8], [9], [10], [11], and the class of models based on the unpaired data (that is, such a one-to-one correspondence is not required as shown in Figure 2(d)) [12], [13], [14], [15], [16].…”

“…In [9], the authors proposed an effective model called CalliGAN by incorporating some kinds of component information such as strokes of Chinese characters, where several auxiliary network modules were introduced to extract these kinds of component information. In the recent paper [10], the authors proposed a novel GAN-based image translation model by integrating the skeleton information for the generation of Chinese brush handwriting font. In [11], the authors proposed a font generation method that learns localized styles, namely component-wise style representations.…”

“…In [11], the authors proposed a font generation method that learns localized styles, namely component-wise style representations. Although the performance of this kind of model based on paired data in the literature [7], [8], [9], [10], [11] is impressive, the collection of extensive paired samples is generally labour-intensive and expensive. Particularly, in some font generation tasks such as the generation of ancient calligraphy fonts [4], [5], it is hard to yield extensive paired samples.…”