Deep Graphical Feature Learning for Face Sketch Synthesis

Abstract

The exemplar-based face sketch synthesis method generally contains two steps: neighbor selection and reconstruction weight representation. Pixel intensities are widely used as features by most of the existing exemplar-based methods, which lacks of representation ability and robustness to light variations and clutter backgrounds. We present a novel face sketch synthesis method combining generative exemplar-based method and discriminatively trained deep convolutional neural networks (dCNNs) via a deep graphical feature learning framework. Our method works in both two steps by using deep discriminative representations derived from dCNNs. Instead of using it directly, we boost its representation capability by a deep graphical feature learning framework. Finally, the optimal weights of deep representations and optimal reconstruction weights for face sketch synthesis can be obtained simultaneously. With the optimal reconstruction weights, we can synthesize high quality sketches which is robust against light variations and clutter backgrounds. Extensive experiments on public face sketch databases show that our method outperforms state-of-the-art methods, in terms of both synthesis quality and recognition ability.

Semi-supervised Learning for Face Sketch Synthesis in the Wild

“…To tackle this problem, Zhou et al [3] presented the markov weight fields (MWF) model which produces a target sketch patch as a linear combination of K best candidate sketch patches. Considering that patch matching based on traditional image features (e.g., PCA and SIFT) is not robust, a recent method [4] used CNN feature to represent the training patches and computed more accurate combination coefficients. To accelerate the synthesis procedure, Song et al [1] formulated face sketch synthesis as a spatial sketch denoising (SSD) problem, and Wang et al [13] presented an offline random sampling strategy for nearest neighbor selection of patches.…”

“…DGFL [4] FCN [14]Pix2Pix-GAN [28] Cycle-GAN [30] Ours Fig. 4: Sketches generated using different methods.…”

“…They then compose an output sketch using the corresponding sketch patches in the reference set. Although promising results arXiv:1812.04929v2 [cs.CV] 5 May 2019 have been reported [1,3,4,5], these methods have several drawbacks. For example, sketches in Fig.…”

Scoot: A Perceptual Metric for Facial Sketches

“…Distortion Types. In order to provide more diverse distortion types arise from the real synthesis algorithms, in introduce some classical methods [27,31,45,53,55,59,64,75,92,93] to achieve this goal. As shown in Fig.…”

“…To minimize the ambiguity of human ranking, we follow the voting strategy [54] to conduct this experi-linear warping lightness shift noise structural damage shifting contrast change blur component lost ghosting checkerboard Figure 6: Our distortions. These distortions are generated by various real synthesis algorithms [27,31,45,53,55,59,64,75,92,93]. ment (∼152K judgments) through the following stages:…”