MCWESRGAN: Improving Enhanced Super-Resolution Generative Adversarial Network for Satellite Images

Abstract

With the dynamic technological development, we are witnessing a major progress in solutions that allow for the observation of Earth's surface. The constructed satellites are becoming still smaller and lighter. Small satellites (those weighing less than 100kg) have a significant drawback (in terms of observational capabilities). Due to their limitations, the installed optic systems are not perfect. As a result, the quality of the obtained images is lower, including lower resolution, although the satellites move on the Low Earth Orbit. In the case of images lacking a high-resolution counterpart, the spatial resolution of the imagery can be improved using single-image super-resolution (SISR) algorithms. In this article, we present an SISR solution based on a new network called MCWESRGAN, which is a modification of the popular ESRGAN network. We propose a novel strategy that introduces a multi-column discriminator model. The generator model is trained using Wasserstein loss. The introduced modifications enable a tenfold reduction in the training time of the network. The proposed algorithm is verified using images obtained from space, aerial imagery, and the Dataset for Object deTection in Aerial Images (DOTA) database. A set of evaluation methods for superresolution (SR) images is proposed to verify the results. These evaluation methods indicate areas that are poorly estimated by the algorithm. Furthermore, as part of the conducted experiments, an absolute assessment method for interpretational potential based on the power spectral density of the image (PSD) is proposed, allowing for determining the magnitude of interpretational improvement after applying resolution enhancement algorithms. The conducted research demonstrates that we achieve better qualitative and quantitative results than classical ESRGAN methods and other state-of-the-art (SOTA) approaches.

Modified ESRGAN with Uformer for Video Satellite Imagery Super-Resolution

“…When we were conducting our research on MCWESRGAN (Multicolumn Wasserstein Enhanced Super-Resolution Generative Adversarial Networks) [58], we noted that SR images were characterized by a specific texture. Based on this finding, we decided to search for a method that would improve the quality of images by limiting the occurrence of deformations.…”

“…The diagram of the presented methodology is shown in Figure 1. In order to conduct the tests to determine the quality of the proposed method, the network used as the one responsible for enhancing spatial resolution was the MCWESRGAN network created by the authors, which was described in detail in the publication [58]. The quality of SR images was improved with the use of the Uformer algorithm that employs the locally enhanced window Transformer block.…”