FIOLA: An accelerated pipeline for Fluorescence Imaging OnLine Analysis

Abstract

Optical microscopy methods such as calcium and voltage imaging already enable fast activity readout (30-1000Hz) of large neuronal populations using light. However, the lack of corresponding advances in online algorithms has slowed progress in retrieving information about neural activity during or shortly after an experiment. This technological gap not only prevents the execution of novel real-time closed-loop experiments, but also hampers fast experiment-analysis-theory turnover for high-throughput imaging modalities. The fundamental challenge is to reliably extract neural activity from fluorescence imaging frames at speeds compatible with new indicator dynamics and imaging modalities. To meet these challenges and requirements, we propose a framework for Fluorescence Imaging OnLine Analysis (FIOLA). FIOLA exploits computational graphs and accelerated hardware to preprocess fluorescence imaging movies and extract fluorescence traces at speeds in excess of 300Hz on calcium imaging datasets and at speeds over 400Hz on voltage imaging datasets. Besides, we present the first real-time spike extraction algorithm for voltage imaging data. We evaluate FIOLA on both simulated data and real data, demonstrating reliable and scalable performance. Our methods provide the computational substrate required to interface precisely large neuronal populations and machines in real-time, enabling new applications in neuroprosthetics, brain-machine interfaces, and experimental neuroscience. Moreover, this new set of tools is poised to dramatically shorten the experiment-data-theory cycle by providing immediate feedback on the activity of large neuronal populations at experimental time.

Imaging neuronal voltage beyond the scattering limit

“…This establishes precise localization as a general approach to improve image resolution, even when light diffraction (13,14) is not the primary constraint. In vivo, ALI resolves neurons beyond the capability of human observers, distinguishing it from existing cell finding methods (19,(23)(24)(25)(26)(27)(28)(29). As we demonstrate here, >100 densely labeled neurons can be simultaneously imaged at high speed and signal separation, providing a density and scale of recording unmatched by existing techniques.…”

Two-photon calcium imaging of neuronal activity

“…These would not only facilitate closed-loop and all-optical experiments (jointly with optogenetics 245 ) but also provide immediate feedback on ongoing experiments and improve scalability. The use of GPUs is a promising avenue to exploit machine learning hardware and software developments that can significantly speed up analyses 246 or the development of advanced brain-computer interfaces. Although some recent research is focusing on building brain-computer interfaces with calcium imaging 151 , the performance is still limited because of a lack in speed and precision in extracting neural activity 246 .…”

Review of data processing of functional optical microscopy for neuroscience

“…Initial work is promising in the ability to motion correct 104 and infer calcium activity. 206,207 However, the ability to infer activity and cell shapes completely online for dense neural fields is still an ongoing research direction.…”

“…While basic manual annotation is trivial to move to an online setting, full demixing algorithms that solve many of the aforementioned challenges are still in their nascent stages. Initial work is promising in the ability to motion correct 104 and infer calcium activity 206 , 207 . However, the ability to infer activity and cell shapes completely online for dense neural fields is still an ongoing research direction.…”