Generative Adversarial Network for Image Super-Resolution Combining Texture Loss

Abstract

Objective: Super-resolution reconstruction is an increasingly important area in computer vision. To alleviate the problems that super-resolution reconstruction models based on generative adversarial networks are difficult to train and contain artifacts in reconstruction results, we propose a novel and improved algorithm. Methods: This paper presented TSRGAN (Super-Resolution Generative Adversarial Networks Combining Texture Loss) model which was also based on generative adversarial networks. We redefined the generator network and discriminator network. Firstly, on the network structure, residual dense blocks without excess batch normalization layers were used to form generator network. Visual Geometry Group (VGG)19 network was adopted as the basic framework of discriminator network. Secondly, in the loss function, the weighting of the four loss functions of texture loss, perceptual loss, adversarial loss and content loss was used as the objective function of generator. Texture loss was proposed to encourage local information matching. Perceptual loss was enhanced by employing the features before activation layer to calculate. Adversarial loss was optimized based on WGAN-GP (Wasserstein GAN with Gradient Penalty) theory. Content loss was used to ensure the accuracy of low-frequency information. During the optimization process, the target image information was reconstructed from different angles of high and low frequencies. Results: The experimental results showed that our method made the average Peak Signal to Noise Ratio of reconstructed images reach 27.99 dB and the average Structural Similarity Index reach 0.778 without losing too much speed, which was superior to other comparison algorithms in objective evaluation index. What is more, TSRGAN significantly improved subjective visual evaluations such as brightness information and texture details. We found that it could generate images with more realistic textures and more accurate brightness, which were more in line with human visual evaluation. Conclusions: Our improvements to the network structure could reduce the model’s calculation amount and stabilize the training direction. In addition, the loss function we present for generator could provide stronger supervision for restoring realistic textures and achieving brightness consistency. Experimental results prove the effectiveness and superiority of TSRGAN algorithm.

Generative Adversarial Network for Heritage Image Super Resolution

“…In literature, varieties of advanced GAN models have been proposed to address these issues either by improving the network structure or by integrating modified loss functions in the optimization process. Here, we will discuss some of the advanced GAN models [9][10][11][12][13] by integrating modified loss functions. Conventionally, the loss functions loss GT P and loss Adv are computed either in pixel-space or in feature-space.…”

“…This method produced much faithful reconstruction of HR images but suffered from computational burden. Method in [12] utilized texture loss, perceptual loss, content loss and adversarial loss. Method in [13], integrated image quality based loss function (Quality-loss) along with content, perceptual and adversarial loss for the GB to yield visually pleasing output image.…”

Upsampling Real-Time, Low-Resolution CCTV Videos Using Generative Adversarial Networks

“…Thus, evaluating the error can help to improve the accuracy of the reconstruction. The content loss can be defined as in (11) [37].…”

“…Through the adversarial training both the generator and discriminator improve their performances and converge to the main goal. Adversarial loss can be defined as in (12) [37].…”

“…In our proposed methodology we used the VGG-19 model [38] to extract features. Perceptual Loss is defined below in (13) [37].…”