PCDRN: Progressive Cascade Deep Residual Network for Pansharpening

Abstract

Pansharpening is the process of fusing a low-resolution multispectral (LRMS) image with a high-resolution panchromatic (PAN) image. In the process of pansharpening, the LRMS image is often directly upsampled by a scale of 4, which may result in the loss of high-frequency details in the fused high-resolution multispectral (HRMS) image. To solve this problem, we put forward a novel progressive cascade deep residual network (PCDRN) with two residual subnetworks for pansharpening. The network adjusts the size of an MS image to the size of a PAN image twice and gradually fuses the LRMS image with the PAN image in a coarse-to-fine manner. To prevent an overly-smooth phenomenon and achieve high-quality fusion results, a multitask loss function is defined to train our network. Furthermore, to eliminate checkerboard artifacts in the fusion results, we employ a resize-convolution approach instead of transposed convolution for upsampling LRMS images. Experimental results on the Pléiades and WorldView-3 datasets prove that PCDRN exhibits superior performance compared to other popular pansharpening methods in terms of quantitative and visual assessments.

Research on super-resolution reconstruction of remote sensing images: a comprehensive review

“…(2) Multi-frame remote sensing image SR reconstruction based on deep residual network In deep learning, high-frequency details are often lost due to the difficult learning of nonlinear feature maps during fusion of multi-spectral and panchromatic images, and the spatial resolution of multi-spectral images should be improved. Therefore, Yang et al 78 considered two residuals by gradually cascading them to learn non-linear feature maps from LR multi-spectral and panchromatic images to HR multi-spectral images. Zheng et al 79 used a deep residual network to fuse hyperspectral and panchromatic images for the first time.…”

Detail Information Prior Net for Remote Sensing Image Pansharpening

“…A bidirectional pyramid network [21] extracts features from a PAN image by convolution operations and produces good pansharpening results by subpixel convolutional SR fusion of MS and PAN image features at corresponding scales. The PCDRN [22] method progressively fuses images through ResNets and interpolation based on the scale relationship between MS images and PAN images. The PCDRN method has shown good fusion performance in high-resolution satellite imagery.…”

Variational Bayesian Pansharpening with Super-Gaussian Sparse Image Priors

“…Instead of facing the difficult task of learning the whole image, residual networks [ 42 , 43 ] learn, from upsampled MS and PAN patches, only the details of the MS high-resolution image that are not already in the upsampled MS image and add them to it to obtain the pansharpened image. To adjust the size of the MS image to the size of the PAN one in a coarse-to-fine manner, two residual networks in cascade were set in the so called Progressive Cascade Deep Residual Network (PCDRN) [ 44 ]. In [ 45 ] a multi-scale approach is followed by learning a DNN to upsample each NSCT directional sub-band from the MS and PAN images.…”