Enhanced Resolution of FY4 Remote Sensing Visible Spectrum Images Utilizing Super-Resolution and Transfer Learning Techniques

Abstract

Remote sensing images acquired by the FY4 satellite are crucial for regional cloud monitoring and meteorological services. Inspired by the success of deep learning networks in image superresolution, we applied image super-resolution to FY4 visible spectrum (VIS) images. However, training a robust network directly for FY4 VIS image super-resolution remains challenging due to the limited provision of high resolution FY4 sample data. Here, we propose a super-resolution and transfer learning model, FY4-SR-Net. It is composed of pre-training and fine-tuning models. The pre-training model was developed using a deep residual network and a large number of FY4 A 4km and 1km resolution VIS images as the training data. The knowledge derived from 4 km to 1 km resolution images was incorporated into FY4 B 1 km to 0.25 km resolution VIS images. The FY4-SR-Net is finetuned by incorporating limited 1km and 0.25km resolution panchromatic (PAN) images, and then producing 1km superresolution VIS images of the FY4 satellite. Using the one-day FY4 test dataset for qualitative and quantitative evaluations, the FY4-SR-Net outperformed the classic bicubic interpolation approach with a 16.12% reduction in root mean square error (RMSE) and a 2.97% rise in peak signal-to-noise ratio (PSNR) averages. The structural similarity (SSIM) value average increased by 0.0026. This work provides a new precedent for improving the spatial resolution of FY4 series meteorological satellites, which has important scientific significance and application properties.

An improved generative adversarial network for remote sensing image super‐resolution

“…Zhang et al. [9] proposed a transfer learning method based on pretraining and fine‐tuning. Using the depth residual model, a pretraining network was established to effectively improve the spatial resolution of the image.…”

“…The very deep super-resolution network (VDSR) [8] introduces the structure of residual learning and further deepens the number of network structure layers based on SRCNN, thereby avoiding the problem that the gradient disappears easily when the neural network has many layers. Zhang et al [9] proposed a transfer learning method based on pretraining and fine-tuning. Using the depth residual model, a pretraining network was established to effectively improve the spatial resolution of the image.…”

Time-Series FY4A Datasets for Super-Resolution Benchmarking of Meteorological Satellite Images

“…Ref. [20] have tried the 8-bit FY4A data for super-resolution, but the 16-bit reconstruction is far challenging.…”

A space-time downscaling approach of Fengyun-4A satellite based on deep learning

“…In order to recover more fine texture information and improve the space-time resolution of the image, Christian Ledig et al proposed Super-Resolution Generative Adversarial Network (SRGAN) for image superresolution [14] . Zhang B et al [15] proposed the Super Resolution and Transfer Learning model (FY4-SR-NET). Firstly, a large number of low-resolution visible light images were used to train the deep residual network, and then a small number of high-resolution panchromatic (PAN) images were used to fine-tune the network to obtain high-resolution visible light images.…”