SRFlow: Learning the Super-Resolution Space with Normalizing Flow

Abstract

Super-resolution is an ill-posed problem, since it allows for multiple predictions for a given low-resolution image. This fundamental fact is largely ignored by state-of-the-art deep learning based approaches. These methods instead train a deterministic mapping using combinations of reconstruction and adversarial losses. In this work, we therefore propose SRFlow: a normalizing flow based super-resolution method capable of learning the conditional distribution of the output given the lowresolution input. Our model is trained in a principled manner using a single loss, namely the negative log-likelihood. SRFlow therefore directly accounts for the ill-posed nature of the problem, and learns to predict diverse photo-realistic high-resolution images. Moreover, we utilize the strong image posterior learned by SRFlow to design flexible image manipulation techniques, capable of enhancing super-resolved images by, e.g., transferring content from other images. We perform extensive experiments on faces, as well as on super-resolution in general. SRFlow outperforms state-of-the-art GAN-based approaches in terms of both PSNR and perceptual quality metrics, while allowing for diversity through the exploration of the space of super-resolved solutions. Code: git.io/Jfpyu.

Model Specialization for the Use of ESRGAN on Satellite and Airborne Imagery

“…However, this approach does not allow the analyst to judge the real capacity of the model to create a new image that has not been degraded by bicubic convolution beforehand. Recent research has tried to overcome this difficulty [26,8,27]. However, these new architectures are beyond the scope of our study and should be the subject of further work.…”

“…Image resolution enhancement is called super-resolution (SR) and is currently a very active research topic in EO image analysis [4][5][6] and computer vision in general, as shown in [7]. However, SR is inherently an ill-posed problem [8].…”

“…Image resolution enhancement is called super-resolution (SR) and is currently a very active research topic in EO image analysis [4][5][6] and computer vision in general, as shown in [7]. However, SR is inherently an ill-posed problem [8]. Multi-frame super-resolution (MFSR) uses multiple low-resolution (LR) images to constrain the reconstruction of a high-resolution (HR) image.…”

“…Otherwise, the model exhibits overfitting and is susceptible to overlearning, i.e., no further improvement in performance can be achieved and, indeed, further "tinkering" (experimentation or adjustment) may possibly result in its subsequent deterioration. Indeed, neural network models can be over-parameterized, yet can still correctly predict labels, even if these were assigned randomly [7][8][9]. Overfitting is a known problem in neural network models and deep learning in general, but different methods can allow us to avoid the problem, such as data augmentation by flipping or rotating the images [18,19], the use of a dropout layer that forces the model to work with part of its parameters turned "off" [20], or early stopping of the training phase [21,22].…”

Lightweight Dual-Stream Residual Network for Single Image Super-Resolution

“…Addressing the problem has been proven to be useful in many practical cases, such as medical imaging [1], [2], infrared imaging [3]- [5] and remote sensing imaging [6]- [8]. From mathematical point of view, SISR needs to build a degradation model from high-resolution(HR) images to low-resolution(LR) images and fit the inverse function [9]. The goal of SISR is to generate high-quality super-resolution(SR) images with clear details that are as close as possible to HR images.…”