Learning to Reconstruct Confocal Microscopy Stacks From Single Light Field Images

Vizcaíno, Josué Page; Saltarin, Federico; Belyaev, Yury; Lyck, Ruth; Lasser, Tobias; Favaro, Paolo

doi:10.1109/tci.2021.3097611

Cited by 25 publications

(32 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reconstruction speed has been improved by a number of groups through deep learning solutions. 19,33 We have shown that 3D deconvolution achieves higher spatial signal confinement than synthetic refocusing with axial confinement increasing at high iteration numbers. Therefore, to maximize spatial signal confinement a time-consuming iterative deconvolution technique could be beneficial.…”

Section: Discussionmentioning

confidence: 92%

Comparing synthetic refocusing to deconvolution for the extraction of neuronal calcium transients from light fields

et al. 2022

View full text Add to dashboard Cite

Significance: Light-field microscopy (LFM) enables fast, light-efficient, volumetric imaging of neuronal activity with calcium indicators. Calcium transients differ in temporal signal-to-noise ratio (tSNR) and spatial confinement when extracted from volumes reconstructed by different algorithms.Aim: We evaluated the capabilities and limitations of two light-field reconstruction algorithms for calcium fluorescence imaging.Approach: We acquired light-field image series from neurons either bulk-labeled or filled intracellularly with the red-emitting calcium dye CaSiR-1 in acute mouse brain slices. We compared the tSNR and spatial confinement of calcium signals extracted from volumes reconstructed with synthetic refocusing and Richardson-Lucy three-dimensional deconvolution with and without total variation regularization.Results: Both synthetic refocusing and Richardson-Lucy deconvolution resolved calcium signals from single cells and neuronal dendrites in three dimensions. Increasing deconvolution iteration number improved spatial confinement but reduced tSNR compared with synthetic refocusing. Volumetric light-field imaging did not decrease calcium signal tSNR compared with interleaved, widefield image series acquired in matched planes.Conclusions: LFM enables high-volume rate, volumetric imaging of calcium transients in single cell somata (bulk-labeled) and dendrites (intracellularly loaded). The trade-offs identified for tSNR, spatial confinement, and computational cost indicate which of synthetic refocusing or deconvolution can better realize the scientific requirements of future LFM calcium imaging applications.

show abstract

Section: Discussionmentioning

confidence: 92%

Comparing synthetic refocusing to deconvolution for the extraction of neuronal calcium transients from light fields

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Traditionally, however, this process is computationally very demanding and takes up to several minutes for a single image 36 amounting to many hours or even days computing time for a whole time series, which makes recording of cellular dynamics unattainable. Several AI-based algorithms have been proposed to speed up the deconvolution and enhance performance 18, 37, 38 , that significantly outperform traditional light field processing. To create a neural network for the reconstruction of C. elegans expressing a fluorescent calcium reporter in the body wall muscles, we first trained a NN with synthetic light field data 37 as the input and experimental confocal stacks as the target (Fig.…”

Section: Resultsmentioning

confidence: 99%

Volumetric bioluminescence imaging of cellular dynamics with deep learning based light-field reconstruction

Morales-Curiel

Castro-Olvera

Gonzalez

et al. 2022

Preprint

View full text Add to dashboard Cite

The application of genetically encoded fluorophores for microscopy has afforded one of the biggest revolutions in the biosciences. Bioluminescence microscopy is an appealing alternative to fluorescence microscopy, because it does not depend on external illumination, and consequently does neither produce spurious background autofluorescence, nor perturb intrinsically photosensitive processes in living cells and animals. The low quantum yield of known luciferases, however, limit the acquisition of high signal-noise images of fast biological dynamics. To increase the versatility of bioluminescence microscopy, we present an improved low-light microscope in combination with deep learning methods to increase the signal to noise ratio in extremely photon-starved samples at millisecond exposures for timelapse and volumetric imaging. We apply our method to image subcellular dynamics in mouse embryonic stem cells, the epithelial morphology during zebrafish development, and DAF-16 FoxO transcription factor shuttling from the cytoplasm to the nucleus under external stress. Finally, we concatenate neural networks for denoising and light-field deconvolution to resolve intracellular calcium dynamics in three dimensions of freely moving Caenorhabditis elegans with millisecond exposure times. This technology is cost-effective and has the potential to replace standard optical microscopy where external illumination is prohibitive.

show abstract

“…In part (a), the first three rows show the 2P 3D image used as ground truth, and the reconstruction using two model-based approaches: ISRA and ADMM, respectively. Furthermore, in the next two rows we evaluate the state-of-the-art LFMNet proposed in [10] and we show our approach. We show several slices for different depths.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Note that all methods are affected by scattering as the In our experiments, the LFMNet achieved best performance among competing methods. Since the deep learning methods are trained with a very small dataset compared to the size of the dataset used in [10], [11], [9], their performance may be affected. In contrast, our method is more robust under this adverse condition.…”

Section: Reconstruction Of Structural 3d Images From Ligh Field Imagesmentioning

confidence: 99%

See 1 more Smart Citation

Physics-based Deep Learning for Imaging Neuronal Activity via Two-photon and Light Field Microscopy

Jadan

Howe

Song

et al. 2022

Preprint

View full text Add to dashboard Cite

Light Field Microscopy (LFM) is an imaging technique that offers the opportunity to study fast dynamics in biological systems due to its rapid 3D imaging rate. In particular, it is attractive to analyze neuronal activity in the brain. Unlike scanning-based imaging methods, LFM simultaneously encodes the spatial and angular information of light in a single snapshot. However, LFM is limited by a trade-off between spatial and angular resolution and is affected by scattering at deep layers in the brain tissue. In contrast, two-photon (2P) microscopy is a point-scanning 3D imaging technique that achieves higher spatial resolution, deeper tissue penetration, and reduced scattering effects. However, point-scanning acquisition limits the imaging speed in 2P microscopy and cannot be used to simultaneously monitor the activity of a large population of neurons. This work introduces a physics-driven deep neural network to image neuronal activity in scattering volume tissues using LFM. The architecture of the network is obtained by unfolding the ISTA algorithm and is based on the observation that the neurons in the tissue are sparse. The deep-network architecture is also based on a novel imaging system modeling that uses a linear convolutional neural network and fits the physics of the acquisition process. To achieve the high-quality reconstruction of neuronal activity in 3D brain tissues from temporal sequences of light field (LF) images, we train the network in a semi-supervised manner using generative adversarial networks (GANs). We use the TdTomato indicator to obtain static structural information of the tissue with the microscope operating in 2P scanning modality, representing the target reconstruction quality. We also use additional functional data in LF modality with GCaMP indicators to train the network. Our approach is tested under adverse conditions: limited training data, background noise, and scattering samples. We experimentally show that our method performs better than model-based reconstruction strategies and typical artificial neural networks for imaging neuronal activity in mammalian brain tissue, considering reconstruction quality, generalization to functional imaging, and reconstruction speed.

show abstract

Learning to Reconstruct Confocal Microscopy Stacks From Single Light Field Images

Cited by 25 publications

References 48 publications

Comparing synthetic refocusing to deconvolution for the extraction of neuronal calcium transients from light fields

Comparing synthetic refocusing to deconvolution for the extraction of neuronal calcium transients from light fields

Volumetric bioluminescence imaging of cellular dynamics with deep learning based light-field reconstruction

Physics-based Deep Learning for Imaging Neuronal Activity via Two-photon and Light Field Microscopy

Contact Info

Product

Resources

About