SSNet: Structure-Semantic Net for Chinese typography generation based on image translation

Zhang, Jianwei; Chen, Danni; Han, Guoqiang; Li, Guanzhao; He, Junting; Liu, Zhen-Mei; Ruan, Zhihui

doi:10.1016/j.neucom.2019.08.072

“…It can directly transfer the learned style to a new font instead of relearning the mapping function from the new certain source style to the target style. SSNet [14] held the point that current methods do not take the Chinese semantics and structure into consideration. Instead of directly generating font images, they employed the structure module and semantic module to require the font feature and character semantics, and then combined this information to generate the final target typography.…”

Section: Disentangled Representationsmentioning

confidence: 99%

“…1) We propose a font fusion network to create new font images by fusing the disentangled skeleton shape and stroke style of different complex font images, while the existing approaches [11][12][13][14] focus on font image translation (imitation of existing font images). 2) Aiming at the problem that the existing method [10] cannot be used for the fusion of font images stably, we propose fuzzy supervised learning skill, which can stabilize the training process of GANs by designing fault tolerance factors.…”

Section: Introductionmentioning

confidence: 99%

Disentangled representation learning GANs for generalized and stable font fusion network

Qin

¹

,

Zhang

²

,

Zhou

³

2021

IET Image Processing

View full text Add to dashboard Cite

Automatic generation of calligraphy fonts has attracted broad attention of researchers. However, previous font generation research mainly focused on the known font style imitation based on image to image translation. For poor interpretability, it is hard for deep learning to create new fonts with various font styles and features according to human understanding. To address this issue, the font fusion network based on generative adversarial networks (GANs) and disentangled representation learning is proposed in this paper to generate brand new fonts. It separates font into two understandable disentangled features: stroke style and skeleton shape. According to personal preferences, various new fonts with multiple styles can be generated by fusing the stroke style and skeleton shape of different fonts. First, this task improves the interpretability of deep learning, and is more challenging than simply imitating font styles. Second, considering the robustness of the network, a fuzzy supervised learning skill is proposed to enhance the stability of the fusion of two fonts with considerable discrepancy. Finally, instead of retraining, the authors' trained model can be quickly transferred to other font fusion samples. It improves the efficiency of the model. Qualitative and quantitative results demonstrate that the proposed method is capable of efficiently and stably generating the new font images with multiple styles. The source code and the implementation details of our model are available at https://github.com/Qinmengxi/Fontfusion.

show abstract

“…The stylish Chinese font generation has attracted rising attention within recent years (Lin et al 2016;Cha et al 2020;Chang, Gu, and Zhang 2017;Tian 2017;Kong and Xu 2017;Jiang et al 2017Jiang et al , 2019Chang et al 2018;Chen et al 2019;Wu, Yang, and Hsu 2020;Gao and Wu 2020;Zhang et al 2020), since it has a wide range of applications including but not limited to the automatic generation of artistic Chinese calligraphy (Zhao et al 2020), art font design (Lin et al 2014) and personalized style generation of Chinese characters (Liu, Xu, and Lin 2012).…”

Section: Introductionmentioning

confidence: 99%

StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding

Zeng

¹

,

Chen

²

,

Liu

³

et al. 2021

AAAI

View full text Add to dashboard Cite

The generation of stylish Chinese fonts is an important problem involved in many applications. Most of existing generation methods are based on the deep generative models, particularly, the generative adversarial networks (GAN) based models. However, these deep generative models may suffer from the mode collapse issue, which significantly degrades the diversity and quality of generated results. In this paper, we introduce a one-bit stroke encoding to capture the key mode information of Chinese characters and then incorporate it into CycleGAN, a popular deep generative model for Chinese font generation. As a result we propose an efficient method called StrokeGAN, mainly motivated by the observation that the stroke encoding contains amount of mode information of Chinese characters. In order to reconstruct the one-bit stroke encoding of the associated generated characters, we introduce a stroke-encoding reconstruction loss imposed on the discriminator. Equipped with such one-bit stroke encoding and stroke-encoding reconstruction loss, the mode collapse issue of CycleGAN can be significantly alleviated, with an improved preservation of strokes and diversity of generated characters. The effectiveness of StrokeGAN is demonstrated by a series of generation tasks over nine datasets with different fonts. The numerical results demonstrate that StrokeGAN generally outperforms the state-of-the-art methods in terms of content and recognition accuracies, as well as certain stroke error, and also generates more realistic characters.

show abstract

“…The stylish Chinese font generation has attracted rising attention within recent years (Lin et al 2016;Cha et al 2020;Chang, Gu, and Zhang 2017;Tian 2017;Kong and Xu 2017;Jiang et al 2017Jiang et al , 2019Chang et al 2018;Chen et al 2019;Wu, Yang, and Hsu 2020;Gao and Wu 2020;Zhang et al 2020), since it has a wide range of applications including but not limited to the automatic generation of artistic Chinese calligraphy (Zhao et al 2020), art font design (Lin et al 2014) and personalized style generation of Chinese characters (Liu, Xu, and Lin 2012).…”

Section: Introductionmentioning

confidence: 99%

“…The second category of Chinese font generation methods has been recently studied in (Tian 2017;Chang et al 2018;Chen et al 2019;Gao and Wu 2020;Wu, Yang, and Hsu 2020;Zhang et al 2020) with the development of deep learning (Goodfellow, Bengio, and Courville 2016), particularly the generative adversarial networks (GAN) (Goodfellow et al 2014). Due to the powerful expressivity and approximation ability of deep neural networks, feature extraction and generation procedures can be combined into one procedure, and thus, Chinese font generation methods in the second category can be usually realized in an end-to-end training way.…”

Section: Introductionmentioning

confidence: 99%

StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding

Zeng

¹

,

Chen

²

,

Liu

³

et al. 2020

Preprint

View full text Add to dashboard Cite

The generation of stylish Chinese fonts is an important problem involved in many applications. Most of existing generation methods are based on the deep generative models, particularly, the generative adversarial networks (GAN) based models. However, these deep generative models may suffer from the mode collapse issue, which significantly degrades the diversity and quality of generated results. In this paper, we introduce a one-bit stroke encoding to capture the key mode information of Chinese characters and then incorporate it into CycleGAN, a popular deep generative model for Chinese font generation. As a result we propose an efficient method called StrokeGAN, mainly motivated by the observation that the stroke encoding contains amount of mode information of Chinese characters. In order to reconstruct the one-bit stroke encoding of the associated generated characters, we introduce a stroke-encoding reconstruction loss imposed on the discriminator. Equipped with such one-bit stroke encoding and stroke-encoding reconstruction loss, the mode collapse issue of CycleGAN can be significantly alleviated, with an improved preservation of strokes and diversity of generated characters. The effectiveness of StrokeGAN is demonstrated by a series of generation tasks over nine datasets with different fonts. The numerical results demonstrate that StrokeGAN generally outperforms the state-of-the-art methods in terms of content and recognition accuracies, as well as certain stroke error, and also generates more realistic characters.

show abstract

SSNet: Structure-Semantic Net for Chinese typography generation based on image translation

Cited by 14 publications

References 9 publications

Disentangled representation learning GANs for generalized and stable font fusion network

Disentangled representation learning GANs for generalized and stable font fusion network

StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding

StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding

Contact Info

Product

Resources

About