A compressive hyperspectral video imaging system using a single-pixel detector

Abstract

Capturing fine spatial, spectral, and temporal information of the scene is highly desirable in many applications. However, recording data of such high dimensionality requires significant transmission bandwidth. Current computational imaging methods can partially address this challenge but are still limited in reducing input data throughput. In this paper, we report a video-rate hyperspectral imager based on a single-pixel photodetector which can achieve high-throughput hyperspectral video recording at a low bandwidth. We leverage the insight that 4-dimensional (4D) hyperspectral videos are considerably more compressible than 2D grayscale images. We propose a joint spatial-spectral capturing scheme encoding the scene into highly compressed measurements and obtaining temporal correlation at the same time. Furthermore, we propose a reconstruction method relying on a signal sparsity model in 4D space and a deep learning reconstruction approach greatly accelerating reconstruction. We demonstrate reconstruction of 128 × 128 hyperspectral images with 64 spectral bands at more than 4 frames per second offering a 900× data throughput compared to conventional imaging, which we believe is a first-of-its kind of a single-pixel-based hyperspectral imager.

A Loss-initiated GAN-based Convolutional LSTM Method for Compression and Motion Estimation-Based Objective Enhancement in Images and Videos

“…Previous studies have offered theoretical support for utilising autoencoders (AEs) for visual codecs that aim for a better rate-bending compromise. They have also demonstrated the efficacy of inactive portrayal as an arrangement of packed signals [7,8]. Video encoding employs a real-world link between consecutive edges, whereas picture reduction reduces apparent repetition in spatial areas.…”