Ultra-fast fit-free analysis of complex fluorescence lifetime imaging via deep learning

Abstract

Fluorescence lifetime imaging (FLI) provides unique quantitative information in biomedical and molecular biology studies, but relies on complex data fitting techniques to derive the quantities of interest. Herein, we propose a novel fit-free approach in FLI image formation that is based on Deep Learning (DL) to quantify complex fluorescence decays simultaneously over a whole image and at ultra-fast speeds. Our deep neural network (DNN), named FLI-Net, is designed and model-based trained to provide all lifetime-based parameters that are typically employed in the field. We demonstrate the accuracy and generalizability of FLI-Net by performing quantitative microscopic and preclinical experimental lifetime-based studies across the visible and NIR spectra, as well as across the two main data acquisition technologies. Our results demonstrate that FLI-Net is well suited to quantify complex fluorescence lifetimes, accurately, in real time in cells and intact animals without any parameter settings. Hence, it paves the way to reproducible and quantitative lifetime studies at unprecedented speeds, for improved dissemination and impact of FLI in many important biomedical applications, especially in clinical settings.

DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning

“…Deep learning has proven to be adept at analyzing microscopic data [ 21 – 29 ], especially for single-molecule localization, handling dense fields of emitters over small axial ranges (<1.5 μ m) [ 19 , 30 – 37 ] or sparse emitters spread over larger ranges [ 38 ]. Moreover, an emerging application is to jointly design the optical system alongside the data processing algorithm, enabling end-to-end optimization of both components [ 36 , 39 – 46 ].…”

Learning an optimal PSF-pair for ultra-dense 3D localization microscopy

“…Deep Learning (DL) has excelled in a variety of challenging computational-imaging problems in computer vision, computational photography, medical imaging, and microscopy [24,25]. Within the realm of computational microscopy, DL has been deployed for tasks such as cell segmentation [26], image restoration [27][28][29][30], sample classification [31,32], artificial labelling [33], phase imaging [34][35][36], optical tomography [37], lifetime imaging [38], single-molecule localization [15][16][17][18][19][20][21][22][23][39][40][41][42], aberration correction [43][44][45][46], CryoEM [47], and more [48].…”

DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning

“…Recently, deep learning has proven to be a useful tool for microscopic data analysis [21][22][23][24][25][26], and specifically for localization microscopy [19,[27][28][29][30][31][32][33]. Moreover, an emerging promising application of deep learning is to jointly design the optical system alongside the data processing algorithm, enabling end-to-end optimization of both components [34][35][36][37][38][39][40][41][42].…”