SWCGAN: Generative Adversarial Network Combining Swin Transformer and CNN for Remote Sensing Image Super-Resolution

Abstract

Easy and efficient acquisition of high-resolution remote sensing images is of importance in geographic information systems. Previously, deep neural networks composed of convolutional layers have achieved impressive progress in superresolution reconstruction. However, the inherent problems of the convolutional layer, including the difficulty of modeling the longrange dependency, limit the performance of these networks on super-resolution reconstruction. To address the above problems, we propose a generative adversarial network (GAN) by combining the advantages of the swin transformer and convolutional layers, called SWCGAN. It is different from the previous superresolution models, which are composed of pure convolutional blocks. The essential idea behind the proposed method is to generate high-resolution images by a generator network with a hybrid of convolutional and swin transformer layers and then to use a pure swin transformer discriminator network for adversarial training. In the proposed method, (1) we employ a convolutional layer for shallow feature extraction that can be adapted to flexible input sizes; (2) we further propose the residual dense swin transformer block (RDSTB) to extract deep features for upsampling to generate high-resolution images; and (3) we use a simplified swin transformer as the discriminator for adversarial training. To evaluate the performance of the proposed method, we compare the proposed method with other state-of-the-art methods by utilizing the UCMerced benchmark dataset, and we apply the proposed method to real-world remote sensing images. The results demonstrate that the reconstruction performance of the proposed method outperforms other state-of-the-art methods in most metrics.

An Efficient Hybrid CNN-Transformer Approach for Remote Sensing Super-Resolution

“…Lei et al [38] proposed a Transformer-based Enhancement Network (TransENet), where the transformer is employed to extract features at different stages, and the multi-stage design allows for the fusion of high-dimensional and low-dimensional features. Tu et al [47] combined the Swin Transformer with generative adversarial networks (GANs) to propose SWCGAN, where the generator is composed of both convolution and swin and the discriminator consists solely of the Swin Transformer. Shang et al [48] designed a hybrid-scale hierarchical transformer network (HSTNet) to acquire long-range dependencies and effectively compute the correlations between high-dimensional and low-dimensional features.…”

MFFAGAN: Generative Adversarial Network With Multilevel Feature Fusion Attention Mechanism for Remote Sensing Image Super-Resolution

“…A weighted channel-wise concatenation is used to replace the summation of elements in all skip connections, which further facilitates the combination of high-frequency features, enhancing information integration. Tu et al [35] proposed SWCGAN, which combines Swin Transformer and convolutional layers. The method significantly improves the perceptual quality of the reconstructed HR image by stacking deep feature extraction blocks (RDSTB) and adopting a novel Swin Transformer discriminator.…”

Hybrid-Scale Hierarchical Transformer for Remote Sensing Image Super-Resolution

“…Ye et al [42] proposed a transformer-based super-resolution method for RSIs, and they employed self-attention to establish dependencies relationships within local and global features. Tu et al [43] presented a GAN that draws on the strengths of the CNN and Swin Transformer, termed the SWCGAN. The SWCGAN fully considers the characteristics of large size, a large amount of information, and a strong relevance between pixels required for RSISR.…”