Fast and robust active neuron segmentation in two-photon calcium imaging using spatiotemporal deep learning

Soltanian-Zadeh, Somayyeh; Sahingur, Kaan; Blau, Sarah; Gong, Yiyang; Farsiu, Sina

doi:10.1073/pnas.1812995116

Cited by 111 publications

(137 citation statements)

References 40 publications

Supporting

Mentioning

136

Contrasting

Order By: Relevance

“…Accomplishing this task is important to understand ensemble responses and to improve the visualization of signal integration across multiple brain regions. Although several efforts were able to ensure a somewhat automated processing of these imaging results through classic machine learning techniques ( Mukamel et al, 2009 ; Kaifosh et al, 2014 ; Guan et al, 2018 ), current approaches are now entering the realm of convolutional neural networks, which can be used in a 3D architecture to segment active neurons on different layers of the same imaging target ( Soltanian-Zadeh et al, 2019 ). Also, accessible algorithms for imaging motion correction have been recently developed and can be used to improve the output quality of automated segmentation networks ( Pnevmatikakis and Giovannucci, 2017 ).…”

Section: Introductionmentioning

confidence: 99%

Cell Calcium Imaging as a Reliable Method to Study Neuron–Glial Circuits

2020

View full text Add to dashboard Cite

Complex dynamic cellular networks have been studied in physiological and pathological processes under the light of single-cell calcium imaging (SCCI), a method that correlates functional data based on calcium shifts operated by different intracellular and extracellular mechanisms integrated with their cell phenotypes. From the classic synaptic structure to tripartite astrocytic model or the recent quadripartite microglia added ensemble, as well as other physiological tissues, it is possible to follow how cells signal spatiotemporally to cellular patterns. This methodology has been used broadly due to the universal properties of calcium as a second messenger. In general, at least two types of receptor operate through calcium permeation: a fast-acting ionotropic receptor channel and a slow-activating metabotropic receptor, added to exchangers/transporters/pumps and intracellular Ca 2+ release activated by messengers. These prototypes have gained an enormous amount of information in dynamic signaling circuits. SCCI has also been used as a method to associate phenotypic markers during development and stage transitions in progenitors, stem, vascular cells, neuro- and glioblasts, neurons, astrocytes, oligodendrocytes, and microglia that operate through ion channels, transporters, and receptors. Also, cancer cells or inducible cell lines from human organoids characterized by transition stages are currently being used to model diseases or reconfigure healthy cells in terms of the expression of calcium-binding/permeable molecules and shed light on therapy.

show abstract

Section: Introductionmentioning

confidence: 99%

Cell Calcium Imaging as a Reliable Method to Study Neuron–Glial Circuits

2020

View full text Add to dashboard Cite

show abstract

“…Although it is virtually impossible to completely eliminate the physiological causes of movement artefacts, efficient mechanical, technical and software approaches have been developed to limit their burden and facilitate data interpretation. With the ongoing advances in molecular, optical and genetic tools that are available to access, identify and study biological mechanisms in increasingly demanding physiological environments and conditions, there is no doubt that two‐photon intravital microscopy will occupy a front‐line position in the future of biomedical research (Tomek et al ., 2013; Pfeiffer et al ., 2018; Soltanian‐Zadeh et al ., 2019). Of note is that technical developments related to light access, acquisition, processing speed and signal collection efficiency will increase the amount of data being collected, thereby stressing the need for robust hardware and software solutions to efficiently manipulate the imaging data.…”

Section: Resultsmentioning

confidence: 99%

Multiphoton intravital microscopy in small animals: motion artefact challenges and technical solutions

Soulet

Lamontagne‐Proulx

Aubé

et al. 2020

Journal of Microscopy

View full text Add to dashboard Cite

Summary Since its invention 29 years ago, two‐photon laser‐scanning microscopy has evolved from a promising imaging technique, to an established widely available imaging modality used throughout the biomedical research community. The establishment of two‐photon microscopy as the preferred method for imaging fluorescently labelled cells and structures in living animals can be attributed to the biophysical mechanism by which the generation of fluorescence is accomplished. The use of powerful lasers capable of delivering infrared light pulses within femtosecond intervals, facilitates the nonlinear excitation of fluorescent molecules only at the focal plane and determines by objective lens position. This offers numerous benefits for studies of biological samples at high spatial and temporal resolutions with limited photo‐damage and superior tissue penetration. Indeed, these attributes have established two‐photon microscopy as the ideal method for live‐animal imaging in several areas of biology and have led to a whole new field of study dedicated to imaging biological phenomena in intact tissues and living organisms. However, despite its appealing features, two‐photon intravital microscopy is inherently limited by tissue motion from heartbeat, respiratory cycles, peristalsis, muscle/vascular tone and physiological functions that change tissue geometry. Because these movements impede temporal and spatial resolution, they must be properly addressed to harness the full potential of two‐photon intravital microscopy and enable accurate data analysis and interpretation. In addition, the sources and features of these motion artefacts are varied, sometimes unpredictable and unique to specific organs and multiple complex strategies have previously been devised to address them. This review will discuss these motion artefacts requirement and technical solutions for their correction and after intravital two‐photon microscopy.

show abstract

“…While both work very well in practice, different approaches for detecting neurons in static images or in a short residual buffer could potentially be employed here, e.g. dictionary learning [25], combinatorial clustering [26] or deep neural networks [27,28]. However, these approaches likely come with higher computational cost, and -having been developed for offline processing -would probably need to be modified for data streams, and in the case of neural networks be retrained.…”

Section: Discussionmentioning

confidence: 99%

Online analysis of microendoscopic 1-photon calcium imaging data streams

Friedrich

Giovannucci

Pnevmatikakis

2020

Preprint

View full text Add to dashboard Cite

In-vivo calcium imaging through microendoscopic lenses enables imaging of neuronal populations deep within the brains of freely moving animals. Previously, a constrained matrix factorization approach (CNMF-E) has been suggested to extract single-neuronal activity from microendoscopic data. However, this approach relies on offline batch processing of the entire video data and is demanding both in terms of computing and memory requirements. These drawbacks prevent its applicability to the analysis of large datasets and closed-loop experimental settings. Here we address both issues by introducing two different online algorithms for extracting neuronal activity from streaming microendoscopic data. Our first algorithm presents an online adaptation of the CNMF-E algorithm, which dramatically reduces its memory and computation requirements. Our second algorithm proposes a convolution-based background model for microendoscopic data that enables even faster (real time) processing on GPU hardware. Our approach is modular and can be combined with existing online motion artifact correction and activity deconvolution methods to provide a highly scalable pipeline for microendoscopic data analysis. We apply our algorithms on two previously published typical experimental datasets and show that they yield similar high-quality results as the popular offline approach, but outperform it with regard to computing time and memory requirements. Author summaryCalcium imaging methods enable researchers to measure the activity of genetically-targeted large-scale neuronal subpopulations. Whereas previous methods required the specimen to be stable, e.g. anesthetized or head-fixed, new brain imaging techniques using microendoscopic lenses and miniaturized microscopes have enabled deep brain imaging in freely moving mice.However, the very large background fluctuations, the inevitable movements and distortions of imaging field, and the extensive spatial overlaps of fluorescent signals complicate the goal of efficiently extracting accurate estimates of neural activity from the observed video data. Further, current activity extraction methods are computationally expensive due to the complex background model and are typically applied to imaging data after the experiment is complete. Moreover, in some scenarios it is necessary to perform experiments in real-time and closed-loop -analyzing data on-the-fly to guide the next experimental steps or to control feedback -, and this calls for new methods for accurate real-time processing. Here we address both issues by adapting a popular extraction method to operate online and extend it to utilize GPU hardware 1/24 that enables real time processing. Our algorithms yield similar high-quality results as the original offline approach, but outperform it with regard to computing time and memory requirements. Our results enable faster and scalable analysis, and open the door to new closed-loop experiments in deep brain areas and on freely-moving preparations.

show abstract

Fast and robust active neuron segmentation in two-photon calcium imaging using spatiotemporal deep learning

Cited by 111 publications

References 40 publications

Cell Calcium Imaging as a Reliable Method to Study Neuron–Glial Circuits

Cell Calcium Imaging as a Reliable Method to Study Neuron–Glial Circuits

Multiphoton intravital microscopy in small animals: motion artefact challenges and technical solutions

Online analysis of microendoscopic 1-photon calcium imaging data streams

Contact Info

Product

Resources

About