A hyperspectral projector for simultaneous 3D spatial and hyperspectral imaging via structured illumination

Xu, Yibo; Giljum, Anthony; Kelly, Kevin F.

doi:10.1364/oe.402812

Cited by 20 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Controllable lighting-based systems are based on a traditional 3D reconstruction method that uses extra light sources, such as structured light [15,22,37] and photometric stereo [21,23,27,28]. These systems use a standard RGB camera and observe spectral measurements by temporally changing illumination spectrum.…”

Section: Related Workmentioning

confidence: 99%

“…However, the requirement of a hyperspectral camera makes the system high cost. Some other systems use a standard RGB camera in conjunction with a variable and controllable light source, which emits temporally-changing illuminations to acquire multiband spectral observations [15,[21][22][23]27,28,37]. However, these systems require multiple shots and thus are not applicable to dynamic scenes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection

Li¹,

Monno²,

Okutomi³

2022

Preprint

View full text Add to dashboard Cite

Depth reconstruction and hyperspectral reflectance reconstruction are two active research topics in computer vision and image processing. Conventionally, these two topics have been studied separately using independent imaging setups and there is no existing method which can acquire depth and spectral reflectance simultaneously in one shot without using special hardware. In this paper, we propose a novel single-shot hyperspectral-depth reconstruction method using an off-the-shelf RGB camera and projector. Our method is based on a single color-dot projection, which simultaneously acts as structured light for depth reconstruction and spatially-varying color illuminations for hyperspectral reflectance reconstruction. To jointly reconstruct the depth and the hyperspectral reflectance from a single color-dot image, we propose a novel end-to-end network architecture that effectively incorporates a geometric color-dot pattern loss and a photometric hyperspectral reflectance loss. Through the experiments, we demonstrate that our hyperspectral-depth reconstruction method outperforms the combination of an existing state-of-the-art singleshot hyperspectral reflectance reconstruction method and depth reconstruction method.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection

Li¹,

Monno²,

Okutomi³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Hyperspectral imaging − in optical microscopy is of importance in the study of various submicron physical and chemical phenomena by acquiring the response of a molecule or material to light as a function of both the two-dimensional (2D) spatial position and the one-dimensional (1D) spectral information associated with every spatial position. Hyperspectral imaging has been used, for instance, for the analysis of plasmonic scattering spectra of metal nanostructures, − the detection and localization of fluorescent molecules for biomedical samples, , and in nonlinear optical microscopy. , …”

Section: Introductionmentioning

confidence: 99%

Compressive Hyperspectral Microscopy of Scattering and Fluorescence of Nanoparticles

Giljum

et al. 2022

J. Phys. Chem. C

Self Cite

View full text Add to dashboard Cite

Hyperspectral imaging in optical microscopy is of importance in the study of various submicron physical and chemical phenomena. However, its practical application is still challenging because the additional spectral dimension increases the number of sampling points to be independently measured compared to two-dimensional (2D) imaging. Here, we present a hyperspectral microscopy system through passive illumination approach based on compressive sensing (CS) using a spectrometer with a one-dimensional (1D) detector array and a digital micromirror device (DMD). The illumination is patterned after the sample rather than on it, making this technique compatible with both dark-field and bright-field imaging. The DMD diffraction issue resulting from this approach has been overcome by a novel striped DMD pattern modulation method. In addition, a split pattern method is developed for increasing the spatial resolution when employing the DMD pattern modulation. The efficacy of the system is demonstrated on nanoparticles using two model systems: extended plasmonic metal nanostructures and fluorescent microspheres. The compressive hyperspectral microscopic system provides a fast, high dynamic range, and enhanced signal-to-noise ratio (SNR) platform that yields a powerful and low-cost spectral analytical system to probe the optical properties of a myriad of nanomaterial systems. The system can also be extended to wavelengths beyond the visible spectrum with greatly reduced expense compared to other approaches that use 2D array detectors.

show abstract

“…Recent advances in optical imaging show promising applications in biomedical areas due to the potential correlation between the optical properties of biological tissue and biochemical composition. 1 – 3 Among these technologies, spatial frequency domain imaging (SFDI) has warranted closer attention in that SFDI is effective to derive tissue optical parameters in a noncontact manner. 4 , 5 The key of SFDI is acquiring the absorption and scattering coefficients of the tissues by computing on the pattern image that is captured by the hyperspectral imaging cameras on spatially modulated light.…”

Section: Introductionmentioning

confidence: 99%

Spatial frequency domain imaging technology based on Fourier single-pixel imaging

et al. 2022

View full text Add to dashboard Cite

. Significance: Optical properties (absorption coefficient and scattering coefficient) of tissue are the most critical parameters for disease diagnosis-based optical method. In recent years, researchers proposed spatial frequency domain imaging (SFDI) to quantitatively map tissue optical properties in a broad field of contactless imaging. To solve the limitations in wavebands unsuitable for silicon-based sensor technology, a compressed sensing (CS) algorithm is used to reproduce the original signal by a single-pixel detectors. Currently, the existing single-pixel SFDI method mainly uses a random sampling policy to extract and recover signals in the acquisition stage. However, these methods are memory-hungry and time-consuming, and they cannot generate discernible results under low sampling rate. Explorations on high performance and efficiency single-pixel SFDI are of great significance for clinical application. Aim: Fourier single-pixel imaging can reconstruct signals with less time and space costs and has fewer reconstruction errors. We focus on an SFDI algorithm based on Fourier single-pixel imaging and propose our Fourier single-pixel image-based spatial frequency domain imaging method (FSI-SFDI). Approach: First, we use Fourier single-pixel imaging algorithm to collect and compress signals and SFDI algorithm to generate optical parameters. Given the basis that the main energy of general image signals is concentrated in the range of low frequency of Fourier frequency domain, our FSI-SFDI uses a circular-sampling scheme to sample data points in the low-frequency region. Then, we reconstruct the image details from these points by optimization-based inverse-FFT method. Results: Our algorithm is tested on simulated data. Results show that the root mean square error (RMSE) of optical parameters is lower than 5% when the data reduction is 92%, and it can generate discernible optical parameter image with low sampling rate. We can observe that our FSI-SFDI primarily recovers the optical properties while keeping the RMSE under the upper bound of 4.5% when we use an image with resolution as the example for calculation and analysis. Not only that but also our algorithm consumes less space and time for an image with resolution, the signal reconstruction takes only 1.65 ms, and requires less RAM memory. Compared to CS-SFDI method, our FSI-SFDI can reduce the required number of measurements through optimizing algorithm. Conclusions: Moreover, FSI-SFDI is capable of recovering high-quality resolvable images with lower sampling rate, higher-resolution images with less memory and time consumed than previous CS-SFDI method, which is very promising for clinical data collection and medical analysis.

show abstract

A hyperspectral projector for simultaneous 3D spatial and hyperspectral imaging via structured illumination

Cited by 20 publications

References 23 publications

Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection

Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection

Compressive Hyperspectral Microscopy of Scattering and Fluorescence of Nanoparticles

Spatial frequency domain imaging technology based on Fourier single-pixel imaging

Contact Info

Product

Resources

About