Hyperspectral image super‐resolution under misaligned hybrid camera system

Abstract

Hyperspectral imaging has been widely used for agriculture, astronomy, surveillance, and so on. However, hyperspectral imaging usually suffers from low‐spatial resolution, due to the limited photons in individual bands. Recently, more hyperspectral image super‐resolution methods have been developed by fusing the low‐resolution hyperspectral image and high‐resolution RGB image, but most of them did not consider the misalignment between two input images. In this study, the authors present an effective method to restore a high‐resolution hyperspectral image from the misaligned low‐resolution hyperspectral image and high‐resolution RGB image, which exploits spectral and spatial correlation in hyperspectral and RGB images. Specifically, they employ the spectral sparsity to restore the high‐resolution hyperspectral image on the misaligned part, and then simultaneously employ spectral and spatial structure correlation to restore the high‐resolution hyperspectral image on the aligned area, which can be fused to obtain the high‐quality hyperspectral image restoration under a misaligned hybrid camera system. Experimental results show that the proposed method outperforms the state‐of‐the‐art hyperspectral image super‐resolution methods under a misaligned hybrid camera system in terms of both objective metric and subjective visual quality.

Direct Alignment Of Narrow Field-Of-View Hyperspectral Data And Full-View Rgb Image

“…These works consider geometric alignment models from in-plane translation only [18,19,20] to rotation and projective homography [9]. But while successfully generating high resolution hyperspectral images, they are only robust to very slight misalignment of the input image pairs of similar viewpoint and FoV, i.e.…”

“…There are hyperspectral image alignment algorithms with an RGB image within hyperspectral spatial super resolution methods [8,9]. However, they only deal with very slight misalignments in carefully designed optical systems acquiring both types of images as if they were acquired from the same viewpoint at the same time.…”