Exploiting Joint Sparsity for Pansharpening: The J-SparseFI Algorithm

Abstract

Recently, sparse signal representation of image patches has been explored to solve the pansharpening problem. Although these proposed sparse-reconstruction-based methods lead to promising results, three issues remained unsolved: 1) high computational cost; 2) no consideration given to the possibility of mutually correlated information in different multispectral channels; and 3) requirement that the spectral responses of the panchromatic (Pan) image and the multispectral image cover the same wavelength range, which is not necessarily valid for most sensors. In this paper, we propose a sophisticated sparse image fusion algorithm, which is named "jointly sparse fusion of images" (J-SparseFI). It is based on the earlier proposed sparse fusion of images (SparseFI) algorithm and overcomes the aforementioned three drawbacks of the existing sparse image fusion algorithms. The computational problem is handled by reducing the problem size and by proposing a fully parallelizable scheme. Moreover, J-SparseFI exploits the possible signal structure correlations between multispectral channels by introducing the joint sparsity model (JSM) and sharpening the highly correlated adjacent multispectral channels together. This is done by exploiting the distributed compressive sensing theory that restricts the solution of an underdetermined system by considering an ensemble of signals being jointly sparse. J-SparseFI also offers a practical solution to overcome spectral range mismatch between the Pan and multispectral images. By means of sensor spectral response and channel mutual correlation analysis, the multispectral channels are assigned to primary groups of joint channels, secondary groups of joint channels, and individual channels. Primary groups of joint channels, individual channels, and secondary groups of joint channels are then reconstructed sequentially, by the JSM or by modified SparseFI, using a dictionary trained from the Pan image or previously reconstructed high-resolution multispectral channels. A recipe of how to choose appropriate algorithm parameters, including the most crucial regularization parameter, is provided. The algorithm is evaluated and validated using

A Review of Image Fusion Algorithms Based on the Super-Resolution Paradigm

“…An improved version of [38] has been very recently proposed by the same authors [46]. It exploits the mutual correlation among multispectral channels by introducing the concept of the joint sparsity model (JSM).…”

“…Four SR-based pan-sharpening methods have been selected for performance assessment: SparseFI [38], J-SparseFI [46], SR-D [45] and the method proposed in [37]. The AWLP algorithm [33] has been chosen as a benchmark to compare the new SR-based pan-sharpening methods to a simple, effective and widely-adopted classical pan-sharpening method.…”

“…Objective comparisons are reported from experiments published in [37,38,45,46] and performed on simulated images produced from sensed airborne HySpex images and on true IKONOS and WorldView-2 images.…”

“…Figure 6 shows the WorldView-2-like images simulated from the airborne visible to near-infrared (VNIR) HySpex data acquired over Munich, Germany, in 2012 by the German Space Agency (DLR). Starting from the input 0.75-m HySpex hyperspectral data cube, whose true-color composition is illustrated in Figure 6a, a 3-m synthetic multispectral image in Figure 6a and a 0.75-m Pan image that match the specifications of the WorldView-2 imager in terms of the spectral properties have been simulated [46]. Therefore, the original 0.75-m data can be used as the reference image for the pan-sharpened products.…”

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

“…Several HSI-MSI fusion algorithms have been proposed in the last decades [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. HR HSI can be reconstructed by combining endmember of LR HSI and an abundance of HR MSI.…”