High-speed hyperspectral video acquisition with a dual-camera architecture

Wang, Lizhi; Xiong, Zhiwei; Gao, Dahua; Shi, Guangming; Zeng, Wenjun; Wu, Feng

doi:10.1109/cvpr.2015.7299128

Cited by 81 publications

(30 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this context, Peng et al [2014] propose a denoising method during reconstruction. Wang et al [2015] introduce a dual-camera system, combining information from panchromatic video at a high frame rate with hyperspectral information at a low frame rate; panchromatic information is used to learn an overcomplete dictionary. introduce a spatial-spectral encoding hyperspectral imager, equipped with a diffraction grating.…”

Section: Related Workmentioning

confidence: 99%

“…Similar to previous works [Li et al 2012;Martin et al 2015], we assume that hyperspectral vectors belong to a subspace of hidden representations. However, instead of using predetermined bases (such as discrete cosine transforms or wavelets) or dictionary-based sparse coding Peng et al 2014;Wang et al 2015], we rely on the convolutional autoencoder to decompose input signals into a set of basis vectors and coefficients. Moreover, while common sparse coding approaches usually reconstruct signals by linear combination of the basis functions, the autoencoder allows for nonlinear reconstruction of hyperspectral information, which fits better the nonlinear nature of the problem, and thus leads to better results (see Section 5).…”

Section: Convolutional Autoencodermentioning

confidence: 99%

See 1 more Smart Citation

High-quality hyperspectral reconstruction using a spectral prior

et al. 2017

View full text Add to dashboard Cite

We present a novel hyperspectral image reconstruction algorithm, which overcomes the long-standing tradeoff between spectral accuracy and spatial resolution in existing compressive imaging approaches. Our method consists of two steps: First, we learn nonlinear spectral representations from real-world hyperspectral datasets; for this, we build a convolutional autoencoder which allows reconstructing its own input through its encoder and decoder networks. Second, we introduce a novel optimization method, which jointly regularizes the fidelity of the learned nonlinear spectral representations and the sparsity of gradients in the spatial domain, by means of our new fidelity prior. Our technique can be applied to any existing compressive imaging architecture, and has been thoroughly tested both in simulation, and by building a prototype hyperspectral imaging system. It outperforms the state-of-the-art methods from each architecture, both in terms of spectral accuracy and spatial resolution, while its computational complexity is reduced by two orders of magnitude with respect to sparse coding techniques. Moreover, we present two additional applications of our method: hyperspectral interpolation and demosaicing. Last, we have created a new high-resolution hyperspectral dataset containing sharper images of more spectral variety than existing ones, available through our project website.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Convolutional Autoencodermentioning

confidence: 99%

High-quality hyperspectral reconstruction using a spectral prior

et al. 2017

View full text Add to dashboard Cite

show abstract

“…In some scenarios different types of image sensors or different configurations (e.g., exposure time and frame rate) are required to achieve multimode sensing. Some examples follow: Shanakr et al [18] studied microcamera diversity for computational image composition; Wang et al [58] combined a DSLR camera and low-budget cameras to capture high-quality light field images with low-cost devices; Kinect [59] captured the RGB and depth images by using two types of sensors; and Wang et al [60] achieved high-quality spectral imaging by combining the highresolution RGB sensor and a coded aperture-based spectral camera together. The Large Synoptic Survey Telescope (LSST) [27] uses three types of sensors, i.e., common image sensors, wavefront sensors, and guide sensors to correct the aberration effect caused by the atmospheric disturbance from images.…”

Section: Image Composition and Displaymentioning

confidence: 99%

Parallel cameras

Brady¹,

Pang²,

Li³

et al. 2018

Optica

View full text Add to dashboard Cite

Parallel lens systems and parallel image signal processing enable cost efficient and compact cameras to capture gigapixel scale images. This paper reviews the context of such cameras in the developing field of computational imaging and discusses how parallel architectures impact optical and electronic processing design. Using an array camera operating system initially developed under the Defense Advanced Research Projects Agency Advanced Wide FOV Architectures for Image Reconstruction and Exploitation program, we illustrate the state of parallel camera development with example 100 megapixel videos.

show abstract

“…Recent development in hyperspectral technology seeks faster image captures comparing to the conventional scanningbased techniques [18], [20]. Several attempts have been made for real-time multi-channel capturing [8], [42], [31], [37]. However, these devices are complicated and/or bulky that limits their usefulness.…”

Section: Introductionmentioning

confidence: 99%

Exposure Invariance in Spectral Reconstruction from RGB Images

Lin

Finlayson

2019

cic

View full text Add to dashboard Cite

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images. Moreover, this spectral reconstruction problem (SR) can often be solved with good (low) error. However, these methods are not physically plausible: that is when the recovered spectra are reintegrated with the underlying camera sensitivities, the resulting predicted RGB is not the same as the actual RGB, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most learningbased SR models train for a fixed exposure setting and we show that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that physical plausibility is enforced and the problem resulting from changing exposures is mitigated. Our SR solution improves the state-of-the-art spectral recovery performance under varying exposure conditions while simultaneously ensuring physical plausibility (i.e. the recovered spectra reintegrate to the input RGBs exactly).

show abstract

High-speed hyperspectral video acquisition with a dual-camera architecture

Cited by 81 publications

References 35 publications

High-quality hyperspectral reconstruction using a spectral prior

High-quality hyperspectral reconstruction using a spectral prior

Parallel cameras

Exposure Invariance in Spectral Reconstruction from RGB Images

Contact Info

Product

Resources

About