Hyperspectral and Multispectral Image Fusion Under Spectrally Varying Spatial Blurs – Application to High Dimensional Infrared Astronomical Imaging

Abstract

Hyperspectral imaging has become a significant source of valuable data for astronomers over the past decades. Current instrumental and observing time constraints allow direct acquisition of multispectral images, with high spatial but low spectral resolution, and hyperspectral images, with low spatial but high spectral resolution. To enhance scientific interpretation of the data, we propose a data fusion method which combines the benefits of each image to recover a high spatio-spectral resolution datacube. The proposed inverse problem accounts for the specificities of astronomical instruments, such as spectrally variant blurs. We provide a fast implementation by solving the problem in the frequency domain and in a low-dimensional subspace to efficiently handle the convolution operators as well as the high dimensionality of the data. We conduct experiments on a realistic synthetic dataset of simulated observation of the upcoming James Webb Space Telescope, and we show that our fusion algorithm outperforms state-of-the-art methods commonly used in remote sensing for Earth observation.

Simulated JWST datasets for multispectral and hyperspectral image fusion

“…1) Method: The synthetic scene and the simulated observed NIRCam Imager and NIRSpec IFU images have been generated to assess the performance of a dedicated fusion method we have developed (Guilloteau et al, 2019). We refer the reader to this latter paper for full details about the method, but provide below the main characteristics of the fusion algorithm.…”

“…In the approach advocated by Guilloteau et al (2019), the spectral regularization ϕ spa (•) in (4) relies on the prior assumption that the spectra of the fused image live in a low dimensional subspace whereas the spatial regularization ϕ spa (•) promotes a smooth spatial content. Due to the highdimensionality of the resulting optimization problem, its solution cannot be analytically computed but requires an iterative procedure.…”

“…Due to the highdimensionality of the resulting optimization problem, its solution cannot be analytically computed but requires an iterative procedure. To get a scalable and fast algorithm able to handle realistic measurements, Guilloteau et al (2019) proposed two computational tricks: i) the problem is formulated in the Fourier domain, where the convolution operators H(•) and M(•) can be efficiently implemented and ii) in a preprocessing step, the JWST forward models are computed in the lowerdimensional subspace induced by the spectral regularization, which leads to sparse and easily storable operators. By combining these two tricks, the final algorithmic procedure minimizing J(•) saves about 90% of the computational time with respect to a naive implementation.…”

Coded Aperture Hyperspectral Image Reconstruction

“…In general, spectral data are valuable as they allow extracting relevant information from scenes by considering their spectral properties. This has been shown to have great potential to be applied in numerous fields such as medicine [ 1 , 2 ], astronomy [ 3 ], agriculture [ 4 , 5 ], or surveillance [ 6 ]. Nevertheless, despite being a solid and active field of study, there still exist some limitations that prevent the popularization of the new techniques emerging from hyperspectral technology: Hyperspectral imagers are quite expensive even to this day; There is a lack of accessible hyperspectral image repositories; Processing algorithms designed as extensions of the ones used for processing RGB images could potentially miss certain special characteristics inherent to the spectral properties of the data.…”

Interpretable Deep Image Prior Method Inspired In Linear Mixture Model For Compressed Spectral Image Recovery

“…Compressive spectral imaging (CSI) techniques commonly require prior knoweledge of the scene to solve the ill-posed recovery inverse problem entailed by the compressive acquisition paradigm [1]. Model-based optimization methods incorporate such prior information through hand-crafted regularizers as total-variation [2,3], sparsity [4,5], and lowrankness [6,7].…”