Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics

Abstract

Abstract-Our framework is the synthesis of multispectral images (MS) at higher spatial resolution, which should be as close as possible to those that would have been acquired by the corresponding sensors if they had this high resolution. This synthesis is performed with the help of a high spatial but low spectral resolution image: the panchromatic (Pan) image. The fusion of the Pan and MS images is classically referred as pan-sharpening. A fused product reaches good quality only if the characteristics and differences between input images are taken into account. Dissimilarities existing between these two data sets originate from two causes-different times and different spectral bands of acquisition. Remote sensing physics should be carefully considered while designing the fusion process. Because of the complexity of physics and the large number of unknowns, authors are led to make assumptions to drive their development. Weaknesses and strengths of each reported method are raised and confronted to these physical constraints. The conclusion of this critical survey of literature is that the choice in the assumptions for the development of a method is crucial, with the risk to drastically weaken fusion performance. It is also shown that the Amélioration de la Résolution Spatiale par Injection de Structures concept prevents from introducing spectral distortion into fused products and offers a reliable framework for further developments.

Image Fusion-Based Land Cover Change Detection Using Multi-Temporal High-Resolution Satellite Images

“…However, each of the image fusion methods has its own disadvantages, with CS-based generating a spectral distortion (or colour/radiometric distortion). The spectral distortion is caused by the mismatch between the spectral response of the MS and PAN bands according to the different bandwidths [19,34]. In order to achieve the aim of our study, the change detection performance using this spectral distortion is used in this study.…”

Landsat 15-m Panchromatic-Assisted Downscaling (LPAD) of the 30-m Reflective Wavelength Bands to Sentinel-2 20-m Resolution

“…Hereafter, for consistency, the term downscaling is used throughout this paper. Downscaling using panchromatic bands has been studied for several decades and the detailed critical surveys can be found in [40][41][42]. Such methods are often grouped into component substitution (CS) and multiresolution analysis (MRA) methods [36,41,43,44].…”

A Parallel Computing Paradigm for Pan-Sharpening Algorithms of Remotely Sensed Images on a Multi-Core Computer

“…Last, the replaced data are reversely transformed back to the original data space. The typical algorithms that apply the CS fusion techniques include IHS [2,34,35], PCA [2], Gram-Schmidt [36], and variants, such as generalized IHS (GIHS) [37], GIHS with tradeoff parameter (GIHS-TP) [38], GIHS with fixed weights (GIHS-F) [37], GIHS adaptive (GIHS-A) [39], improved adaptive IHS (IAIHS) [40], Gram-Schmidt Adaptive (GS-A) [39], fast spectral response function (FSRF) [41], CS with a histogram-matched panchromatic band, CS using sensor spectral response [42,43], CS for vegetation enhancement [44], and the FFT-enhanced IHS transform method [45].…”

“…Generally, these hybrid schemes use wavelet transforms to extract the spatial details from the panchromatic image and subsequently apply an orthogonal transform to inject the details into the multispectral image. The sensor spectral response and ground spectral features are introduced into the fusion techniques based on MRA in [43,64].…”