TV-enhanced deep unfolding network for multispectral image demosaicing

Abstract

Multispectral image (MSI) contains a wealth of spatial information as well as spectral information, making it useful in the application of remote sensing, medical sciences, and beyond. However, traditional scanning-based imaging method is limited to low spatial or temporal resolution. Consequently, the reconstruction of high-resolution, clean, and complete MSI serves as an initial process for the numerous applications. This paper presents a novel deep unfolding network for demosaicing spectral mosaic images obtained through multispectral filter array (MSFA) imaging sensors. Concretely, the proposed network is unfolded from an iterative optimization process into an end-to-end training network, which can efficiently integrate the MSFA-based inherent degradation model with the powerful representation capability of deep neural networks. To further improve performance, a total-variation (TV) denoiser is plugged into the proposed network. Through end-to-end training, the hyperparameters within the optimization framework and TV denoiser are jointly optimized with the parameters of the neural network. Simulation results on CAVE and WHU-OHS datasets show that the proposed method outperforms state-of-the-art methods and improves the generalization capabilities to different MSFA settings.