Face Recognition via Multi-Level 3D-GAN Colorization

Abstract

Rapid development in sketch-to-image translation methods boosts the investigation procedure in law enforcement agencies. But, the large modality gap between manually generated sketches makes this task challenging.Generative adversarial network (GAN) and encoder-decoder approach are usually incorporated to accomplish sketchto-image generation with promising results. This paper targets the sketch-to-image translation with heterogeneous face angles and lighting effects using a multi-level conditional generative adversarial network. The proposed multilevel cGAN work in four different phases. Three independent cGANs' networks are incorporated separately into each stage, followed by a CNN classifier. The Adam stochastic gradient descent mechanism was used for training with a learning rate of 0.0002 and momentum estimates β1 and β2 as 0.5 and 0.999, respectively. The multi-level 3Dconvolutional architecture help to preserve spatial facial attributes and pixel-level details. The 3D convolution and deconvolution guide the G1, G2 and G3 to use additional features and attributes for encoding and decoding. This helps to preserve the direction, postures of targeted image attributes and special relationships among the whole image's features. The proposed framework process the 3D-Convolution and 3D-Deconvolution using vectorization. This process takes the same time as 2D convolution but extracts more features and facial attributes. We used pre-trained ResNet-50, ResNet-101, and Mobile-Net to classify generated high-resolution images from sketches. We have also developed, and state-of-the-art Pakistani Politicians Face-sketch Dataset (PPFD) for experimental purposes. Result reveals that the proposed cGAN model's framework outperforms with respect to Accuracy, Structural similarity index measure (SSIM), Signal to noise ratio (SNR), and Peak signal-to-noise ratio (PSNR).

Adversarial sketch-photo transformation for enhanced face recognition accuracy: a systematic analysis and evaluation

“…Due to the adversarial nature of this process, the generator network may acquire the ability to produce images that are difficult to identify from genuine photographs while yet maintaining the identification information. Adversarial sketch-photo transformation approaches have been proven in recent research to greatly enhance the accuracy and realism of face sketch recognition models [18]. These techniques can help create more lifelike photographs from sketched facial features, which has potential uses in areas like facial animation and repair.…”