Deep Learning Enhanced Hyperspectral Fluorescence Lifetime Imaging

Abstract

Acquiring dense high-dimensional optical data in biological applications remains a challenge due to the very low levels of light typically encountered. Single pixel imaging methodologies enable improved detection efficiency in such conditions but are still limited by relatively slow acquisition times. Here, we propose a Deep Learning framework, NetFLICS-CR, which enables fast hyperspectral lifetime imaging for in vivo applications at enhanced resolution, acquisition and processing speeds, without the need of experimental training datasets. NetFLICS-CR reconstructs intensity and lifetime images at 128x128 pixels over 16 spectral channels while reducing the current acquisition times from ~2.5 hours at 50% compression to ~3 minutes at 99% compression when using a single-pixel Hyperspectral Macroscopic Fluorescence Lifetime Imaging (HMFLI) system. The potential of the technique is demonstrated in silico, in vitro and in vivo through the monitoring of receptor-ligand interactions in mice liver and bladder and further imaging of intracellular drug delivery of the clinical drug Trastuzumab in live animals bearing HER2-positive breast tumor xenografts.

Review of Fluorescence Lifetime Imaging Microscopy (FLIM) Data Analysis Using Machine Learning

“…They showed that FPFLI performed where MLE failed to estimate lifetimes in low photon counts. Ochoa et al [65] proposed NetFLICS-CR, where a compressed ratio (CR) block is added to the NetFLICS [61] model. This CR block reduces the input dimension.…”

Spectral and lifetime fluorescence unmixing via deep learning

“…fluorophores currently known to possess short lifetime values (sub-nanosecond) and correspondingly high analytic complexity. Lastly, we present hyperspectral lifetime unmixing results for two in vivo datasets as acquired in [9]: 1) Trastuzumab (TZM) AF700/AF750-conjugated FRET pair, for an athymic nude mouse bearing a tumor xenograft and imaged 76 hours postinjection and 2) Transferrin (Tf) AF700/AF750-conjugated FRET pair to distinguish between mouse liver and bladder.…”