ESRGAN+ : Further Improving Enhanced Super-Resolution Generative Adversarial Network

Abstract

Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) is a perceptual-driven approach for single image super resolution that is able to produce photorealistic images. Despite the visual quality of these generated images, there is still room for improvement. In this fashion, the model is extended to further improve the perceptual quality of the images. We have designed a novel block to replace the one used by the original ESRGAN. Moreover, we introduce noise inputs to the generator network in order to exploit stochastic variation. The resulting images present more realistic textures.

Single Image Super-Resolution Restoration of TGO CaSSIS Colour Images: Demonstration with Perseverance Rover Landing Site and Mars Science Targets

“…GANs provide a state-of-the-art framework for producing high-quality and "photorealistic" SRR images. Recent GAN variations [92][93][94] have been focusing on optimisations of the original residual architecture of the generator network [85,91] and/or on better modelling of the perceptual loss, in order to improve the visual quality of the SRR results. In this work, we based our model on the ESRGAN architecture [92] due to its solid performance on real-world images.…”

“…In this work, we based our model on the ESRGAN architecture [92] due to its solid performance on real-world images. Inspired by the adaptive weighted learning process proposed in AWSRN [88] and the optimisations introduced in ESRGANplus [93], we propose an Adaptive Weighted RRDB with noise inputs (AWRRDB) to replace the original RRDB basic block in ESRGAN for more effective and efficient residual learning. Moreover, we use a multi-scale reconstruction scheme [88] based on subpixel-shuffling [80,85] to replace the up-sampling layers used in ESRGAN to make full use of both low-frequency and high-frequency residuals while avoiding the checkerboard patterned artefacts from using up-sampling layers [91].…”

“…The second modification to the RRDB structure is adding a Gaussian noise input after each DB. The additive Gaussian noise inputs were demonstrated as being useful in [93] in terms of adding stochastic variation to the generator network, while keeping their effects very localised, i.e., without changing the global perception of the images. Note that 3 of the 11 weights are scaling factors for the additive Gaussian noise.…”

“…Note that 3 of the 11 weights are scaling factors for the additive Gaussian noise. There was another potential improvement to the DB structure, which was also proposed in [93], called Residual DB (RDB), by adding a residual every two layers to augment the generator network capacity. However, we found the improvement of using RDB is marginal compared to DB in AWRRDB, and therefore we keep the ESRGAN's design of DB in AWRRDB.…”

Multiple Auxiliary Networks for Single Blind Image Deblurring

“…Recently, deep learning and machine learning have shown the potential ability and achieved outstanding performance in many fields [5][6][7][8][9][10], especially some image-to-image tasks such as semantic segmentation [11,12], image superresolution [13,14]. Inspired by this, works most recently tend to train an end-to-end CNN [14][15][16][17][18] for single image deblurring and achieve more effective results than previously established methods.…”