Real‐world super‐resolution of face‐images from surveillance cameras

Abstract

Most existing face image Super-Resolution (SR) methods assume that the Low-Resolution (LR) images were artificially downsampled from High-Resolution (HR) images with bicubic interpolation. This operation changes the natural image characteristics and reduces noise. Hence, SR methods trained on such data most often fail to produce good results when applied to real LR images. To solve this problem, a novel framework for the generation of realistic LR/HR training pairs is proposed. The framework estimates realistic blur kernels, noise distributions, and JPEG compression artifacts to generate LR images with similar image characteristics as the ones in the source domain. This allows to train an SR model using high-quality face images as Ground-Truth (GT). For better perceptual quality, a Generative Adversarial Network (GAN) based SR model is used, where the commonly used VGG-loss [1] is exchanged with LPIPS-loss [2]. Experimental results on both real and artificially corrupted face images show that our method results in more detailed reconstructions with less noise compared to the existing State-of-the-Art (SoTA) methods. In addition, it is shown that the traditional non-reference Image Quality Assessment (IQA) methods fail to capture this improvement and demonstrate that the more recent NIMA metric [3] correlates better with human perception via Mean Opinion Rank (MOR).

Compressed Video Quality Assessment for Super-Resolution: A Benchmark and a Quality Metric

“…LPIPS [32] is a metric that uses deep features of various neural networks to compare images. It additionally serves as a loss function for some SR models [3].…”

Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior

“…SPGAN [30] designed a supervised pixel-wise GAN whose discriminator was used to output the same resolution matrix as the input image and proposed a supervised pixel-wise adversarial loss. RWSR [44] proposed a novel method to synthesize low-quality images for face super-resolution and used the ESRGAN [18] model as the main backbone SR model, which introduced a variety of loss functions to improve image quality. Super-FAN [4] first proposed an end-to-end model that addressed face superresolution and alignment via integrating a sub-network for face alignment through heat map regression and optimizing a heatmap loss.…”

Design of a Low-Power Super-Resolution Architecture for Virtual Reality Wearable Devices

“…Super-Resolution (SR) [2] is a well-known effective technique to reconstruct high-resolution images starting from low-resolution ones. Such a characteristic is very useful in a lot of real-word applications, including video surveillance [3], remote sensing [4] and autonomous driving [5], where processing high-quality images is fundamental to detect and classify specific behaviors. In the context of VR/AR, using SR models based on Convolutional Neural Networks (CNNs) has a dual purpose.…”