Ultra-Resolving Face Images by Discriminative Generative Networks

Abstract

Conventional face super-resolution methods, also known as face hallucination, are limited up to 2 ∼ 4× scaling factors where 4 ∼ 16 additional pixels are estimated for each given pixel. Besides, they become very fragile when the input low-resolution image size is too small that only little information is available in the input image. To address these shortcomings, we present a discriminative generative network that can ultra-resolve a very low resolution face image of size 16 × 16 pixels to its 8× larger version by reconstructing 64 pixels from a single pixel. We introduce a pixel-wise 2 regularization term to the generative model and exploit the feedback of the discriminative network to make the upsampled face images more similar to real ones. In our framework, the discriminative network learns the essential constituent parts of the faces and the generative network blends these parts in the most accurate fashion to the input image. Since only frontal and ordinary aligned images are used in training, our method can ultra-resolve a wide range of very low-resolution images directly regardless of pose and facial expression variations. Our extensive experimental evaluations demonstrate that the presented ultra-resolution by discriminative generative networks (UR-DGN) achieves more appealing results than the state-of-the-art.

To Learn Image Super-Resolution, Use a GAN to Learn How to Do Image Degradation First

“…Face super-resolution is super-resolution applied to faces. Similarly to image super-resolution, the vast majority of face superresolution methods [18][19][20][21][22][23] are based on a paired setting for training and evaluation which is typically done on frontal datasets (e.g. CelebA [24], Helen [25], LFW [26], BioID [27]).…”

“…Rather than directly generating the HR image, the method of [20] proposes to combine CNNs with the Wavelet Transform for predicting a series of corresponding wavelet coefficients. The recent work of [22] is a GAN-based approach similar to the one proposed in [2]. In [18], a two-step decoder-encoder-decoder architecture is proposed also incorporating a spatial transformer network to undo face misalignment.…”

Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement

“…In [65], a Bi-channel CNN is proposed to integrate the input image and face representation for prediction. In [61,22], the GAN framework is applied to hallucinate LR face images. However, the network generates highresolution images from random noise in [22] while the face hallucination task is to tackle a specific input image.…”

Distribution Discrepancy Maximization for Image Privacy Preserving

“…Works in (Oh et al 2016; R, R, and V 2016) employed deep recognition models to successfully defeat common image obfuscation algorithms such as Pixelation, Blurring and P3. In addition, DNN also shows outstanding ability to restore images from severe noise and low-resolution, such as methods in (Mao, Shen, and Yang 2016;Yu and Porikli 2016), which can also be used as a pre-processing step for decrypting. The experimental results of these methods reveal that well-designed deep learning methods can still recognize the sensitive information of encrypted images.…”