Hyperspectral Imaging With Random Printed Mask

Zhao, Yuanyuan; Guo, Hui; Ma, Zhan; Cao, Xun; Yue, Tao; Hu, Xuemei

doi:10.1109/cvpr.2019.01039

Cited by 25 publications

(21 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But, there is the overhead of decompressing the signal. Examples include multi-spectral color filter array [13], coded aperture [6,17,18], diffractive gratings [28], digital micro-mirror device [34] and most recently random printed mask [47]. Other problems inherent in compressive sensing are the need for specialized optics and the inherent trade-off between the number of sensors and/or light sensitivity and the spatial resolution.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Physically Plausible Spectral Reconstruction from RGB Images

Lin

Finlayson

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images, and this spectral reconstruction problem (SR) can often be solved with good (low) error. However, little attention has been paid on whether these models' behavior can adhere to physics. We show that the leading CNN method introduces unexpected 'colorimetric errors', which means the recovered spectra do not reproduce ground-truth RGBs, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most CNN models over-fit to fixed exposure and we demonstrate that this can result in poor performance when exposure varies. In this paper we show how CNN learning can be extended so that the physical plausibility of SR is enforced. Remarkably, our physically plausible CNN solutions advance both spectral and colorimetric performance of the original network, while the application of data augmentation trades off the network performance for model stability against varying exposure.

show abstract

Section: Related Workmentioning

confidence: 99%

“…However, these devices are complicated and/or bulky that limits their usefulness. Other designs deploy novel optical components with specialized post-processing algorithms [13,18,6,17,28,34,47]. But, these devices trade off spatial resolution and/or light sensitivity.…”

Section: Introductionmentioning

confidence: 99%

Physically Plausible Spectral Reconstruction from RGB Images

Lin

Finlayson

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

show abstract

“…But, there is the overhead of decompressing the signal. Examples include multi-spectral color filter array [12], coded aperture [6], [16], [17], diffractive gratings [27], digital micro-mirror device [33] and most recently random printed mask [47]. Other problems inherent in compressive sensing are the need for specialized optics and the inherent trade-off between the number of sensors and/or light sensitivity and the spatial resolution.…”

Section: Related Workmentioning

confidence: 99%

“…However, these devices are complicated and/or bulky that limits their usefulness. Other designs deploy novel optical components with specialized post-processing algorithms [12], [17], [6], [16], [27], [33], [47]. But, these devices trade off spatial resolution and/or light sensitivity for faster capturing speed.…”

Section: Introductionmentioning

confidence: 99%

Exposure Invariance in Spectral Reconstruction from RGB Images

Lin

Finlayson

2019

cic

View full text Add to dashboard Cite

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images. Moreover, this spectral reconstruction problem (SR) can often be solved with good (low) error. However, these methods are not physically plausible: that is when the recovered spectra are reintegrated with the underlying camera sensitivities, the resulting predicted RGB is not the same as the actual RGB, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most learningbased SR models train for a fixed exposure setting and we show that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that physical plausibility is enforced and the problem resulting from changing exposures is mitigated. Our SR solution improves the state-of-the-art spectral recovery performance under varying exposure conditions while simultaneously ensuring physical plausibility (i.e. the recovered spectra reintegrate to the input RGBs exactly).

show abstract

“…However, none of these spectral reconstruction methods assume the raw response images are mosaicked, thus cannot be applied to our problem. A more recent literature [30] utilized random printed masks as filter arrays and reconstructed multispectral images using an end-to-end network.…”

Section: Related Workmentioning

confidence: 99%

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Xie

et al. 2019

Sensors

View full text Add to dashboard Cite

Multispectral filter array (MSFA)-based imaging is a compact, practical technique for snapshot spectral image capturing and reconstruction. The imaging and reconstruction quality is highly influenced by the spectral sensitivities and spatial arrangement of channels on MSFAs, and the used reconstruction method. In order to design a MSFA with high imaging capacity, we propose a sparse representation based approach to optimize spectral sensitivities and spatial arrangement of MSFAs. The proposed approach first overall models the various errors associated with spectral reconstruction, and then uses a global heuristic searching method to optimize MSFAs via minimizing the estimated error of MSFAs. Our MSFA optimization method can select filters from off-the-shelf candidate filter sets while assigning the selected filters to the designed MSFA. Experimental results on three datasets show that the proposed method is more efficient, flexible, and can design MSFAs with lower spectral construction errors when compared with existing state-of-the-art methods. The MSFAs designed by our method show better performance than others even using different spectral reconstruction methods.

show abstract

Hyperspectral Imaging With Random Printed Mask

Cited by 25 publications

References 25 publications

Physically Plausible Spectral Reconstruction from RGB Images

Physically Plausible Spectral Reconstruction from RGB Images

Exposure Invariance in Spectral Reconstruction from RGB Images

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Contact Info

Product

Resources

About