Deep-Learning Supervised Snapshot Compressive Imaging Enabled by an End-to-End Adaptive Neural Network

Marquez, Miguel; Lai, Yingming; Liu, Xianglei; Jiang, Cheng; Zhang, Shian; Argüello, Henry; Liang, Jinyang

doi:10.1109/jstsp.2022.3172592

Cited by 16 publications

(18 citation statements)

References 83 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The time-sheared view preserves temporal information via a compressed-sensing acquisition. Then, an algorithm, built upon the plug-and-play alternating direction method of multipliers [8,9], is used to reconstruct the video, from which the extracted lifetime distribution is converted to a temperature map. Using the core/shell NaGdF4:Er 3+ ,Yb 3+ /NaGdF4 UCNPs as the lifetimebased temperature indicators [Fig.…”

Section: Resultsmentioning

confidence: 99%

Single-shot compressed ultrahigh-speed imaging for fast wide-field upconversion luminescence lifetime thermometry

Liang

2023

Ultrafast Nonlinear Imaging and Spectroscopy XI

Self Cite

View full text Add to dashboard Cite

We develop an optical temperature mapping modality, termed single-shot photoluminescence lifetime imaging thermometry (SPLIT). Synergistically combining dual-view optical streak imaging with compressed sensing, SPLIT records wide-field luminescence decay of Er 3+ , Yb 3+ co-doped NaGdF4 upconverting nanoparticles in real time, from which a lifetime-based 2D temperature map is obtained in a single exposure. Largely advancing existing optical thermometry techniques in detection capabilities, SPLIT enables longitudinal 2D temperature monitoring beneath a thin scattering medium and dynamic temperature tracking of a moving biological sample at single-cell resolution. SPLIT will be applied to early melanoma diagnosis and extended to the near-infrared spectral region.

show abstract

Section: Resultsmentioning

confidence: 99%

Single-shot compressed ultrahigh-speed imaging for fast wide-field upconversion luminescence lifetime thermometry

Liang

2023

Ultrafast Nonlinear Imaging and Spectroscopy XI

Self Cite

View full text Add to dashboard Cite

show abstract

“…In software, the dual-view PnP-ADMM algorithm provides a powerful modular structure, which allows separated optimization of individual sub-optimization problems with an advanced denoising algorithm to generate high-quality image restoration results. In the future, neural network based learning methods [17,18] can be used to supply an end-to-end image reconstruction with a high-fidelity and a faster reconstruction at the cost of the flexibility of data in both size and sequence depth.…”

Section: Discussionmentioning

confidence: 99%

Single-shot photoluminescence lifetime imaging for fast wide-field optical thermometry

Liu

Skripka

Lai

et al. 2023

High-Speed Biomedical Imaging and Spectroscopy VIII

Self Cite

View full text Add to dashboard Cite

Photoluminescence lifetime imaging of upconverting nanoparticles is increasingly featured in recent progress in optical thermometry. Despite remarkable advances in photoluminescent temperature indicators, existing optical instruments lack the ability of wide-field photoluminescence lifetime imaging in real time, thus falling short in dynamic temperature mapping. Here, we have developed single-shot photoluminescence lifetime imaging thermometry (SPLIT), which is developed from a compressed-sensing ultrahigh-speed imaging paradigm. Using the core/shell NaGdF4:Er3+,Yb3+/NaGdF4 upconverting nanoparticles as the lifetime-based temperature indicators, we apply SPLIT in longitudinal wide-field temperature monitoring beneath a thin scattering medium. SPLIT also enables video-rate temperature mapping of a moving biological sample at single-cell resolution.

show abstract

“…Schematic of deep-learning supervised snapshot compressive imaging using the deep high-dimensional adaptive net (D-HAN). Adapted from [15].…”

Section: Structure Of D-hanmentioning

confidence: 99%

“…The upper arm, corresponding to 𝜇 ̃−1 𝚽 𝑇 [𝜇 ̃𝐈 + 𝚽𝚽 𝑇 ] −1 𝚽, is composed of the direct sensing operator as a first layer followed by an inverse and transpose sensing operator along with four 2D convolutional+ReLU layers. The bottom arm, which corresponds to 𝜇 ̃−1 has three 2D convolutional+ReLU layers [15]. The outputs of both arms are subtracted and given as the input to the U-net in the D-HAN that reflects the second equation in Eq.…”

Section: Structure Of D-hanmentioning

confidence: 99%

See 1 more Smart Citation

An end-to-end adaptive neural network for process-aware snapshot compressive temporal imaging

Lai

Marquez

Liu

et al. 2023

High-Speed Biomedical Imaging and Spectroscopy VIII

Self Cite

View full text Add to dashboard Cite

Learning-based compressed sensing algorithms are popularly used for recovering the underlying datacube of snapshot compressive temporal imaging (SCTI), which is a novel technique for recording temporal data in a single exposure. Despite providing fast processing and high reconstruction performance, most deep-learning approaches are merely considered a substitute for analytical-modeling-based reconstruction methods. In addition, these methods often presume the ideal behaviors of optical instruments neglecting any deviation in the encoding and shearing processes. Consequently, these approaches provide little feedback to evaluate SCTI’s hardware performance, which limits the quality and robustness of reconstruction. To overcome these limitations, we develop a new end-to-end convolutional neural network—termed the deep high-dimensional adaptive net (D-HAN)—that provides multi-faceted process-aware supervision to an SCTI system. The D-HAN includes three joint stages: four dense layers for shearing estimation, a set of parallel layers emulating the closed-form solution of SCTI’s inverse problem, and a U-net structure that works as a filtering step. In system design, the D-HAN optimizes the coded aperture and establishes SCTI’s sensing geometry. In image reconstruction, D-HAN senses the shearing operation and retrieves a three-dimensional scene. D-HAN-supervised SCTI is experimentally validated using compressed optical-streaking ultrahigh-speed photography to image the animation of a rotating spinner at an imaging speed of 20 thousand frames per second. The D-HAN is expected to improve the reliability and stability of a variety of snapshot compressive imaging systems.

show abstract

Deep-Learning Supervised Snapshot Compressive Imaging Enabled by an End-to-End Adaptive Neural Network

Cited by 16 publications

References 83 publications

Single-shot compressed ultrahigh-speed imaging for fast wide-field upconversion luminescence lifetime thermometry

Single-shot compressed ultrahigh-speed imaging for fast wide-field upconversion luminescence lifetime thermometry

Single-shot photoluminescence lifetime imaging for fast wide-field optical thermometry

An end-to-end adaptive neural network for process-aware snapshot compressive temporal imaging

Contact Info

Product

Resources

About