Compressive spectral image reconstruction using deep prior and low-rank tensor representation

Abstract

Compressive spectral imaging (CSI) has emerged as an alternative spectral image acquisition technology, which reduces the number of measurements at the cost of requiring a recovery process. In general, the reconstruction methods are based on hand-crafted priors used as regularizers in optimization algorithms or recent deep neural networks employed as an image generator to learn a non-linear mapping from the low-dimensional compressed measurements to the image space. However, these data-driven methods need many spectral images to obtain good performance. In this work, a deep recovery framework for CSI without training data is presented. The proposed method is based on the fact that the structure of some deep neural networks and an appropriated low-dimensional structure are sufficient to impose a structure of the underlying spectral image from CSI. We analyzed the low-dimension structure via the Tucker representation, modeled in the first net layer. The proposed scheme is obtained by minimizing the ℓ 2 -norm distance between the compressive measurements and the predicted measurements, and the desired recovered spectral image is formed just before the forward operator. Simulated and experimental results verify the effectiveness of the proposed method.

DeepTensor: Low-Rank Tensor Decomposition with Deep Network Priors

“…A related but different approach to DeepTensor is the work by Bacca et al [41], who identified that hyperspectral images (which are modeled as 3D tensors) can be represented as the output of a single generative network equipped with 3D convolutional kernels. The work by Bacca et al [41] is a promising framework for solving inverse problems in hyperspectral imaging, but are not aimed at low-rank matrix factorization -which is the focus of this paper. The key difference between their work and DeepTensor is that the input to their 3D network is a low-rank Tucker tensor; in contrast we output a low-rank Tucker tensor.…”

“…2D TV results were obtained by a TV penalty on each slice along z-direction. Bacca et al [41] results were obtained by representing input as a rank-1000 PARAFAC tensor and using an untrained 2D network which output 56 channels. 2D TV + PARAFAC results were obtained with self-supervised learning by representing the volume via rank-1000 PARAFAC decomposition, and a TV penalty on each slice along z-axis.…”

JR2net: A Joint Non-Linear Representation and Recovery Network for Compressive Spectral Imaging

“…However, these hand-crafted methods require expert knowledge of the scene to select which prior is more appropriate for this spectral scene. Consequently, they do not represent the wide variety and non-linearity of spectral image representations [27].…”

“…These methods are based on iterative techniques that replace the hand-crafted prior with a deep neural network (DNN) used to learn a deep prior of the spectral images. Non-data-driven approaches, [27,28] proposed untrained DNN as a deep generative model (DGM) where the input of the network (denoted as latent space) passes through convolution operators to generate the image recovery. The weights of untrained DNN aim to minimize the Euclidean distance between the forward sensing operator of the DNN output and the compressed measurement.…”

Deep Coded Aperture Design: An End-to-End Approach for Computational Imaging Tasks

“…compressive imaging systems (CIS) modulate the spatial field acquiring a set of multiplexed versions to recover a gray-scale image [17]; coded polarization imaging systems (CPI) modulate the polarization field trough a micropolarizer array [21]; coded depth imaging systems (CDI) acquire depth-dependent blurred versions of the scene [20]; coded diffraction patterns (CDP) modulate the phase and amplitude of the scene [7]; and compressive light-field systems (CLF) modulate the spatialangular information and acquire 2D light-field projections [8]; Figure 1 b. lists the above mentioned optical coding systems and relates them with the corresponding application field. The performance of computational imaging (CI) tasks from projected encoded measurements such as, recovery [22], segmentation [23], classification [24], detection [25] and parameter estimation [26], depends on both, the computational processing method and the sensing protocol [27]. Hence, a line of research focuses on computational processing methods from projected encoded measurements, commonly based on optimization problems using prior knowledge through a regularizer that penalizes the main cost function.…”