A Joint Training Model for Face Sketch Synthesis

Abstract

The exemplar-based method is most frequently used in face sketch synthesis because of its efficiency in representing the nonlinear mapping between face photos and sketches. However, the sketches synthesized by existing exemplar-based methods suffer from block artifacts and blur effects. In addition, most exemplar-based methods ignore the training sketches in the weight representation process. To improve synthesis performance, a novel joint training model is proposed in this paper, taking sketches into consideration. First, we construct the joint training photo and sketch by concatenating the original photo and its sketch with a high-pass filtered image of their corresponding sketch. Then, an offline random sampling strategy is adopted for each test photo patch to select the joint training photo and sketch patches in the neighboring region. Finally, a novel locality constraint is designed to calculate the reconstruction weight, allowing the synthesized sketches to have more detailed information. Extensive experimental results on public datasets show the superiority of the proposed joint training model, both from subjective perceptual and the FaceNet-based face recognition objective evaluation, compared to existing state-of-the-art sketch synthesis methods.

Adversarial sketch-photo transformation for enhanced face recognition accuracy: a systematic analysis and evaluation

“…The GAN with gradient penalty [24] uses a Wasserstein generative adversarial network with gradient penalty to synthesize high-quality face photos from face sketches. The approach comprises adversarial training of a generator network and a discriminator network to discriminate between created photographs and actual photos, while the generator network minimizes the Wasserstein distance between the distributions of the two.…”

Efficient Generation of Multiple Sketch Styles Using a Single Network

“…Generally, data-driven methods [9]- [12], [25], [26] have two steps. For a given test photo patch, they search for similar photo patches and their corresponding sketch patches from training data.…”

“…Wang et al [10] proposed random sampling strategy in place of similar photo patch search and employed a locality constraint to model the distinct correlations between the test photo patch and sampled photo patches while computing the weights of linear combination. Based on [10], Wan and Lee [25] proposed a joint training model to consider the training sketches during the weight computation process and modified locality constraint. However, aforementioned methods showed limited performance, yielding serious blur and artifacts in generated face sketches.…”

“…Face sketch synthesis methods can be broadly classified into data-driven methods and model-driven methods [6]. Data-driven methods synthesize a sketch from the linear combination of the training sketch patches [9]- [12], [25], [26] while model-driven methods learn a mathematical function to map a photo to a sketch [13], [14], [27], [28]. Recently, GAN based sketch synthesis methods, which can be considered as model-driven methods, have shown remarkable results [6]- [8].…”

Special Features on Intelligent Imaging and Analysis

“…As an application of image analysis, face sketch synthesis has been tested by Wan and Lee [1], using the joint training method on face photos and sketches. Thus, more detailed information can be recorded in the synthesized sketches.…”