Hybrid-Scale Hierarchical Transformer for Remote Sensing Image Super-Resolution

Shang, Jianrun; Gao, Mingliang; Li, Qilei; Pan, Jinfeng; Zou, Guofeng; Jeon, Gwanggil

doi:10.3390/rs15133442

Cited by 8 publications

(4 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tu et al [47] combined the Swin Transformer with generative adversarial networks (GANs) to propose SWCGAN, where the generator is composed of both convolution and swin and the discriminator consists solely of the Swin Transformer. Shang et al [48] designed a hybrid-scale hierarchical transformer network (HSTNet) to acquire long-range dependencies and effectively compute the correlations between high-dimensional and low-dimensional features. Wang et al [36] created a lightweight convolution called the contextual transformation layer (CTL) to replace 3 × 3 convolutions, which can efficiently extract rich contextual features.…”

Section: Sisr Methods Of Remote-sensing Imagesmentioning

confidence: 99%

An Efficient Hybrid CNN-Transformer Approach for Remote Sensing Super-Resolution

Zhang,

Tan,

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Transformer models have great potential in the field of remote sensing super-resolution (SR) due to their excellent self-attention mechanisms. However, transformer models are prone to overfitting because of their large number of parameters, especially with the typically small remote sensing datasets. Additionally, the reliance of transformer-based SR models on convolution-based upsampling often leads to mismatched semantic information. To tackle these challenges, we propose an efficient super-resolution hybrid network (EHNet) based on the encoder composed of our designed lightweight convolution module and the decoder composed of an improved swin transformer. The encoder, featuring our novel Lightweight Feature Extraction Block (LFEB), employs a more efficient convolution method than depthwise separable convolution based on depthwise convolution. Our LFEB also integrates a Cross Stage Partial structure for enhanced feature extraction. In terms of the decoder, based on the swin transformer, we innovatively propose a sequence-based upsample block (SUB) for the first time, which directly uses the sequence of tokens in the transformer to focus on semantic information through the MLP layer, which enhances the feature expression ability of the model and improves the reconstruction accuracy. Experiments show that EHNet’s PSNR on UCMerced and AID datasets obtains a SOTA performance of 28.02 and 29.44, respectively, and is also visually better than other existing methods. Its 2.64 M parameters effectively balance model efficiency and computational demands.

show abstract

Section: Sisr Methods Of Remote-sensing Imagesmentioning

confidence: 99%

An Efficient Hybrid CNN-Transformer Approach for Remote Sensing Super-Resolution

Zhang,

Tan,

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…The emergence of Vision Transformer (VIT) [21] introduced Transformer to the CV domain, achieving performance beyond CNNs on large datasets. Currently, there have been many works combining CNNs with Transformers for image super-resolution [22][23][24][25][26][27]. These methods use a CNN as a shallow feature extractor and Transformer for deep feature extraction, combining the local feature extraction capability of the CNN and the global modeling capability of Transformer to further improve the quality of SR images.…”

Section: Deep Learning For Image Super-resolutionmentioning

confidence: 99%

“…These methods use a CNN as a shallow feature extractor and Transformer for deep feature extraction, combining the local feature extraction capability of the CNN and the global modeling capability of Transformer to further improve the quality of SR images. Although Transformer can effectively compensate for the localization of CNN, in addition to the local-global learning capability, multi-scale feature learning is equally important for the super-resolution task of remote sensing images [17,18,27]. Unfortunately, Transformer does not have the ability of multi-scale feature learning.…”

Section: Deep Learning For Image Super-resolutionmentioning

confidence: 99%

Landsat-8 to Sentinel-2 Satellite Imagery Super-Resolution-Based Multiscale Dilated Transformer Generative Adversarial Networks

Wang,

Zhang,

Yang

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

Image super-resolution (SR) techniques can improve the spatial resolution of remote sensing images to provide more feature details and information, which is important for a wide range of remote sensing applications, including land use/cover classification (LUCC). Convolutional neural networks (CNNs) have achieved impressive results in the field of image SR, but the inherent localization of convolution limits the performance of CNN-based SR models. Therefore, we propose a new method, namely, the dilated Transformer generative adversarial network (DTGAN) for the SR of multispectral remote sensing images. DTGAN combines the local focus of CNNs with the global perspective of Transformers to better capture both local and global features in remote sensing images. We introduce dilated convolutions into the self-attention computation of Transformers to control the network’s focus on different scales of image features. This enhancement improves the network’s ability to reconstruct details at various scales in the images. SR imagery provides richer surface information and reduces ambiguity for the LUCC task, thereby enhancing the accuracy of LUCC. Our work comprises two main stages: remote sensing image SR and LUCC. In the SR stage, we conducted comprehensive experiments on Landsat-8 (L8) and Sentinel-2 (S2) remote sensing datasets. The results indicate that DTGAN generates super-resolution (SR) images with minimal computation. Additionally, it outperforms other methods in terms of the spectral angle mapper (SAM) and learned perceptual image patch similarity (LPIPS) metrics, as well as visual quality. In the LUCC stage, DTGAN was used to generate SR images of areas outside the training samples, and then the SR imagery was used in the LUCC task. The results indicated a significant improvement in the accuracy of LUCC based on SR imagery compared to low-resolution (LR) LUCC maps. Specifically, there were enhancements of 0.130 in precision, 0.178 in recall, and 0.157 in the F1-score.

show abstract

“…Super-resolution (SR) [11] [12] [13] [14] [15] [16] [17] [18] can reconstruct the HR image that is most similar to the original LR image by using the LR image captured and the prior knowledge learned from the sample library, which can effectively enhance the resolution of low-quality image and recover the image feature details. The use of SR method to improve the resolution of observation targets has always been the focus of remote sensing research.…”

mentioning

confidence: 99%

Structure-Texture Dual Preserving for Remote Sensing Image Super Resolution

Zhao,

Lu,

Zhang

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Most of the existing remote sensing image Super-Resolution (SR) methods based on deep learning tend to learn the mapping from low-resolution (LR) images to High-Resolution (HR) images directly. But they ignore the potential structure and texture consistency of LR and HR spaces, which cause the loss of high-frequency information and produce artifacts. A structure-texture dual preserving (STP) method is proposed to solve this problem and generate pleasing details. Specifically, we propose a novel edge prior enhancement strategy that uses the edges of LR images and the proposed interactive supervised attention module (ISAM) to guide SR reconstruction. First, we introduce the LR edge map as an a prior structural expression for SR reconstruction, which further enhances the SR process with edge preservation capability. In addition, to obtain finer texture edge information, we propose a novel ISAM in order to correct the initial LR edge map with high frequency information. By introducing LR edges and ISAM-corrected HR edges, we build LR-HR edge mapping to preserve the consistency of LR and HR edge structure and texture, which provides supervised information for SR reconstruction. Finally, we explore the salient features of the image and its edges in the ascending space, and restored the difference between LR and HR images by residual and dense learning. A large number of experimental results on Draper and NWPU-RESISC45 datesets show that our model is superior to several advanced SR algorithms in both objective and subjective image quality.

show abstract

Hybrid-Scale Hierarchical Transformer for Remote Sensing Image Super-Resolution

Cited by 8 publications

References 58 publications

An Efficient Hybrid CNN-Transformer Approach for Remote Sensing Super-Resolution

An Efficient Hybrid CNN-Transformer Approach for Remote Sensing Super-Resolution

Landsat-8 to Sentinel-2 Satellite Imagery Super-Resolution-Based Multiscale Dilated Transformer Generative Adversarial Networks

Structure-Texture Dual Preserving for Remote Sensing Image Super Resolution

Contact Info

Product

Resources

About