DD-Net: spectral imaging from a monochromatic dispersed and diffused snapshot

Abstract

We propose a snapshot spectral imaging method for the visible spectral range using a single monochromatic camera equipped with a two-dimensional (2D) binary-encoded phase diffuser placed at the pupil of the imaging lens and by resorting to deep learning (DL) algorithms for signal reconstruction. While spectral imaging was shown to be feasible using two cameras equipped with a single, one-dimensional (1D) binary diffuser and compressed sensing (CS) algorithms [Appl. Opt. 59, 7853 (2020).APOPAI0003-693510.1364/AO.395541], the suggested diffuser design expands the optical response and creates optical spatial and spectral encoding along both dimensions of the image sensor. To recover the spatial and spectral information from the dispersed and diffused (DD) monochromatic snapshot, we developed novel DL algorithms, dubbed DD-Nets, which are tailored to the unique response of the optical system, which includes either a 1D or a 2D diffuser. High-quality reconstructions of the spectral cube in simulation and lab experiments are presented for system configurations consisting of a single monochromatic camera with either a 1D or a 2D diffuser. We demonstrate that the suggested system configuration with the 2D diffuser outperforms system configurations with a 1D diffuser that utilize either DL-based or CS-based algorithms for the reconstruction of the spectral cube.

Eliminating Temporal Illumination Variations in Whisk-broom Hyperspectral Imaging

“…This method was able to perform hyperspectral imaging with 1440 × 960 × 25 spatial and spectral resolution with a fabricated DOE attached to a DSLR sensor. Hauser et al (2020) presented a deep learning-based technique for the reconstruction of a spectral cube measuring 256 × 256 × 29 in the 420-700 nm visible spectral range from dispersed and diffused (DD) monochromatic snapshots captured by a single monochromatic camera equipped with a 2-D separable, binary phase diffuser. Dun et al (2020) presented a joint learning algorithm with a diffractive achromats design and an image recovery neural network.…”

Dual-Channel Switchable Metasurface Filters for Compact Spectral Imaging with Deep Compressive Reconstruction

“…They offer faster and more accurate reconstruction compared to iterative approaches, thanks to the strong fitting ability of deep-learning models that alleviates the high computation costs. Consequently, various deep neural network architectures, such as autoencoders [53], convolutional neural networks [54][55][56][57][58], generative adversarial networks [59], transformers [60], and others, have been utilized for spectral reconstruction. Additionally, the entire reconstruction process can be substituted with a neural network, and end-to-end (E2E) reconstruction allows for the sending of measurements into a deep neural network that directly outputs the reconstruction results.…”