GF-CSTNet: A Method for Pan-Sharpening Remote Sensing Images by Integrating CSPNet and Transformer

Abstract

To tackle challenges like limited receptive fields and inadequate feature extraction encountered in conventional ConvolutionalNeural Networks (CNNs), this paper introduces a remote sensing image sharpening approach named GF-CSTNet. Themethod draws inspiration from the FusionNet architecture and combines CSPNet with Transformer for improved performance.Firstly, this method utilizes guided filtering to enhance the acquired remote sensing image data. The CSPNet and Transformerstructures are then combined to further enhance fusion performance by leveraging their respective advantages. Subsequently, aRep-Conv2Former method is designed to streamline attention and extract diverse receptive field features through a multi-scaleconvolution modulator block. Simultaneously, a reparameterization module is constructed to integrate the multiple branchesgenerated during training into a unified branch during inference, thereby optimizing the model’s inference speed. Furthermore,a residual learning module incorporating attention has been devised to augment the modeling and feature extraction capabilitiesof images. Finally, to tackle the overfitting issue that emerges during training, weight decay has been incorporated into the lossfunction. Experimental results obtained from the GaoFen-2 and WorldView-3 datasets demonstrate the effectiveness of theproposed GF-CSTNet approach. It effectively extracts detailed information from images while avoiding the problem of spectraldistortion.