Rapid detection and recognition of whole brain activity in a freely behaving Caenorhabditis elegans

Abstract

Advanced volumetric imaging methods and genetically encoded activity indicators have permitted a comprehensive characterization of whole brain activity at single neuron resolution in Caenorhabditis elegans. The constant motion and deformation of the nematode nervous system, however, impose a great challenge for consistent identification of densely packed neurons in a behaving animal. Here, we propose a cascade solution for long-term and rapid recognition of head ganglion neurons in a freely moving C. elegans. First, potential neuronal regions from a stack of fluorescence images are detected by a deep learning algorithm. Second, 2-dimensional neuronal regions are fused into 3-dimensional neuron entities. Third, by exploiting the neuronal density distribution surrounding a neuron and relative positional information between neurons, a multi-class artificial neural network transforms engineered neuronal feature vectors into digital neuronal identities. With a small number of training samples, our bottom-up approach is able to process each volume—1024 × 1024 × 18 in voxels—in less than 1 second and achieves an accuracy of 91% in neuronal detection and above 80% in neuronal tracking over a long video recording. Our work represents a step towards rapid and fully automated algorithms for decoding whole brain activity underlying naturalistic behaviors.

Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration

“…The video has been centered and rigidly rotated to maintain a consistent orientation of the worm, but no further straightening has been done. With only a few manually annotated reference frames (78 mins/frame on average with the provided GUI), ZephIR already achieves state-ofthe-art accuracy as reported in recently published works on neuron tracking [11,14,23,46] (Fig 3B , Table 1), where accuracy is measured as the average percentage of neurons correctly tracked, and a neuron is considered correctly tracked if the tracked coordinate is within the volume of the neuron as identified via a manual annotator. First three median frames determined via k-medoids clustering and recommended for annotation as reference frames clearly represent the principal postures that are repeatedly sampled during locomotion (Fig 3A).…”

“…Neuronal activity is calculated as the average ratio of GCaMP and RFP fluorescence intensities of the 9 brightest pixels in a 3x7x7 volume (roughly the size of the cell nucleus in the image) centered around the tracked neuron coordinates after masking out pixels that overlap with the volumes around 5 nearest In order to verify and benchmark the accuracy of our method, we apply the same workflow to a similar dataset of a freely moving worm provided by Nguyen, et al [14], a publicly available dataset with ground-truth results provided that has already been used as a benchmark for other recently published algorithms. [11,14,23,46] Since this dataset is similar in behavior, motion, and imaging conditions, we may reuse the same parameters and follow the same workflow as the previously discussed C. elegans dataset. Doing so, we are again able to achieve state-of-the-art accuracy (84.0%) with only a few reference frames, and adding additional partial annotations increases our top accuracy further (94.48%) (Fig 3E).…”

“…Each volume has been centered and rotated but no further straightening has been done. Since this particular dataset has also been used to benchmark a number of recently developed algorithms, we may also compare ZephIR's accuracy with Neuron Registration Vector Encoding (NeRVE) [14], fast Deep Neural Correspondence (fDNC) [11], 3DeeCellTracker [23], and CeNDeR [46]. https://doi.org/10.1371/journal.pcbi.1012075.g003…”

“…Thus, this approach rarely reduces the quantity of additional training data required for such applications [4,11]. Approaches that successfully reduce training data must make hard assumptions about the underlying structure via direct parameter reduction, regularization, or data augmentation [12][13][14][15]. (3) Typical applications of convolutional neural networks analyze images composed of many discriminable textures that fill the image space [16].…”

Unifying community-wide whole-brain imaging datasets enables robust automated neuron identification and reveals determinants of neuron positioning inC. elegans

“…27,28,25,29,30,31 However, none of them have achieved widespread adoption, due at least in part to their incompatibility with different microscopy data formats and low performance on data acquired from different labs. Automatic approaches to the complementary problem of tracking neurons across video frames have achieved some generalized performance across various datasets 32,33 , but so far there have not been efforts to perform similar training and benchmarking for automatic cell identification. In order to build automatic approaches that are robust, accurate, and generalizable, there is a critical need for a standardized format and compatible tools trained and benchmarked on a consolidated corpus of data that reflects the heterogeneity of microscopy equipment, experimental conditions, and protocols across labs.…”