The desire to understand how the brain generates and patterns behavior has driven rapid methodological innovation in tools to quantify natural animal behavior. While advances in deep learning and computer vision have enabled markerless pose estimation in individual animals, extending these to multiple animals presents unique challenges for studies of social behaviors or animals in their natural environments. Here we present Social LEAP Estimates Animal Poses (SLEAP), a machine learning system for multi-animal pose tracking. This system enables versatile workflows for data labeling, model training and inference on previously unseen data. SLEAP features an accessible graphical user interface, a standardized data model, a reproducible configuration system, over 30 model architectures, two approaches to part grouping and two approaches to identity tracking. We applied SLEAP to seven datasets across flies, bees, mice and gerbils to systematically evaluate each approach and architecture, and we compare it with other existing approaches. SLEAP achieves greater accuracy and speeds of more than 800 frames per second, with latencies of less than 3.5 ms at full 1,024 × 1,024 image resolution. This makes SLEAP usable for real-time applications, which we demonstrate by controlling the behavior of one animal on the basis of the tracking and detection of social interactions with another animal.

Connections between neurons can be mapped by acquiring and analyzing electron microscopic (EM) brain images. In recent years, this approach has been applied to chunks of brains to reconstruct local connectivity maps that are highly informative, yet inadequate for understanding brain function more globally. Here, we present the first neuronal wiring diagram of a whole adult brain, containing 5x10^7 chemical synapses between ~130,000 neurons reconstructed from a female Drosophila melanogaster. The resource also incorporates annotations of cell classes and types, nerves, hemilineages, and predictions of neurotransmitter identities. Data products are available by download, programmatic access, and interactive browsing and made interoperable with other fly data resources. We show how to derive a projectome, a map of projections between regions, from the connectome. We demonstrate the tracing of synaptic pathways and the analysis of information flow from inputs (sensory and ascending neurons) to outputs (motor, endocrine, and descending neurons), across both hemispheres, and between the central brain and the optic lobes. Tracing from a subset of photoreceptors all the way to descending motor pathways illustrates how structure can uncover putative circuit mechanisms underlying sensorimotor behaviors. The technologies and open ecosystem of the FlyWire Consortium set the stage for future large-scale connectome projects in other species.

The forthcoming assembly of the adult Drosophila melanogaster central brain connectome, containing over 125,000 neurons and 50 million synaptic connections, provides a template for examining sensory processing throughout the brain. Here, we create a leaky integrate-and-fire computational model of the entire Drosophila brain, based on neural connectivity and neurotransmitter identity, to study circuit properties of feeding and grooming behaviors. We show that activation of sugar-sensing or water-sensing gustatory neurons in the computational model accurately predicts neurons that respond to tastes and are required for feeding initiation. Computational activation of neurons in the feeding region of the Drosophila brain predicts those that elicit motor neuron firing, a testable hypothesis that we validate by optogenetic activation and behavioral studies. Moreover, computational activation of different classes of gustatory neurons makes accurate predictions of how multiple taste modalities interact, providing circuit-level insight into aversive and appetitive taste processing. Our computational model predicts that the sugar and water pathways form a partially shared appetitive feeding initiation pathway, which our calcium imaging and behavioral experiments confirm. Additionally, we applied this model to mechanosensory circuits and found that computational activation of mechanosensory neurons predicts activation of a small set of neurons comprising the antennal grooming circuit that do not overlap with gustatory circuits, and accurately describes the circuit response upon activation of different mechanosensory subtypes. Our results demonstrate that modeling brain circuits purely from connectivity and predicted neurotransmitter identity generates experimentally testable hypotheses and can accurately describe complete sensorimotor transformations.

In the version of this article initially published, there were errors in the y-axis labels in Fig. 3j. From top down, the first two labels, now reading "Python" and "R", were transposed. The correction has been made in the HTML and PDF versions of the article.