Self-supervised spectral super-resolution for a fast hyperspectral and multispectral image fusion

Abstract

Hyperspectral-multispectral image fusion (HSI-MSI Fusion) for enhancing resolution of hyperspectral images is a hot topic in remote sensing. An important category of approaches for HSI-MSI Fusion is based on deep learning. The main challenges in deep learning based fusion methods include the lack of training data, poor generalization to various datasets, and high computational costs. This paper suggests a new approach to tackle these difficulties by introducing an innovative technique for HSI-MSI fusion. The proposed method involves training a tiny deep neural network that can reconstruct high-resolution hyperspectral images through spectral super-resolution of high-resolution multispectral images. This method does not require high resolution training data and they are artificially generated based on the spatial degradation model of the input observation images. Therefore, the problems of data scarcity and poor generalization are addressed, and also the computational burden is significantly reduced. After conducting thorough experiments, it was found that the proposed method provides promising results. The source code of this method is available at https://github.com/rajaei-arash/SSSR-HSI-MSI-Fusion .