DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning

Abstract

Localization microscopy is an imaging technique in which the positions of individual point emitters (e.g. fluorescent molecules) are precisely determined from their images. This is a key ingredient in single/multiple-particle-tracking and super-resolution microscopy. Localization in three-dimensions (3D) can be performed by modifying the image that a point-source creates on the camera, namely, the point-spread function (PSF). The PSF is engineered to vary distinctively with emitter depth, using additional optical elements. However, localizing multiple adjacent emitters in 3D poses a significant algorithmic challenge, due to the lateral overlap of their PSFs. Here, we train a neural network to localize multiple emitters with densely overlapping PSFs over a large axial range. Furthermore, we then use the network to design the optimal PSF for the multi-emitter case. We demonstrate our approach experimentally with super-resolution reconstructions of mitochondria and volumetric imaging of fluorescently labeled telomeres in cells.

Data science in cell imaging

“…It has also provided solutions to other, less-routine, computational tasks. For example, image restoration algorithms attempt to enhance image quality by inferring high-quality images from low-quality data (Weigert et al, 2018) using a variety of strategies, such as by taking advantage of structural redundancy in an image to reconstruct high-quality super-resolution images from under-sampled localization microscopy data (Ouyang et al, 2018), or by performing point spread function engineering for single-molecule localization (Nehme et al, 2020). Other applications include the inference of intracellular organelle localization from labelfree images and the mapping of different cell microscopy modalities onto one another (Christiansen et al, 2018;Ounkomol et al, 2018), with potential applications including high-content screening (Cheng et al, 2021) and the prediction of the functional cell state, such as stages of the cell cycle or disease progression (Buggenthin et al, 2017;Eulenberg et al, 2017;Yang et al, 2020;Zaritsky et al, 2020 preprint).…”

Characterizing Locus Specific Chromatin Structure and Dynamics with Correlative Conventional and Super Resolution imaging in living cells

“…Our study lays the foundation for further refinement and optimization of the presented correlative conventional fluorescence and PALM imaging technique and its application to study a myriad of different chromatin loci. For instance, extending this technique to 3D to track loci for longer periods of time and to extend the lifetime of the conventional signal would extend the imaging time of loci and result in an improved spatial and temporal resolution 15,28,44,53,54 . In addition, orthogonal RNA binding proteins and/or CRISPR-Cas proteins could be applied to enable simultaneous multicolor imaging of different chromatin regions 14,[18][19][20] .…”

Smart solutions for automated imaging

“…Overlapping signals from fluorophores can confound the analysis of STORM images, particularly for 3D imaging, and this is typically resolved by imaging fewer fluorophores per round -a tactic that precludes high-speed imaging. Yoav Shechtman's team at the Technion in Haifa, Israel, employed a deep-learning-based approach to overcome this difficult scenario 4 . They used their algorithm to engineer the point-spread function (PSF)the representation of a fluorophore's signal as generated by a microscope's detection system -to enable better discrimination of individual signals in 3D.…”