The dorsal raphe (DR) constitutes a major serotonergic input to the forebrain and modulates diverse functions and brain states, including mood, anxiety, and sensory and motor functions. Most functional studies to date have treated DR serotonin neurons as a single population. Using viral-genetic methods, we found that subcortical- and cortical-projecting serotonin neurons have distinct cell-body distributions within the DR and differentially co-express a vesicular glutamate transporter. Further, amygdala- and frontal-cortex-projecting DR serotonin neurons have largely complementary whole-brain collateralization patterns, receive biased inputs from presynaptic partners, and exhibit opposite responses to aversive stimuli. Gain- and loss-of-function experiments suggest that amygdala-projecting DR serotonin neurons promote anxiety-like behavior, whereas frontal-cortex-projecting neurons promote active coping in the face of challenge. These results provide compelling evidence that the DR serotonin system contains parallel sub-systems that differ in input and output connectivity, physiological response properties, and behavioral functions.

Serotonin neurons of the dorsal and median raphe nuclei (DR, MR) collectively innervate the entire forebrain and midbrain, modulating diverse physiology and behavior. To gain a fundamental understanding of their molecular heterogeneity, we used plate-based single-cell RNA-sequencing to generate a comprehensive dataset comprising eleven transcriptomically distinct serotonin neuron clusters. Systematic in situ hybridization mapped specific clusters to the principal DR, caudal DR, or MR. These transcriptomic clusters differentially express a rich repertoire of neuropeptides, receptors, ion channels, and transcription factors. We generated novel intersectional viral-genetic tools to access specific subpopulations. Whole-brain axonal projection mapping revealed that DR serotonin neurons co-expressing vesicular glutamate transporter-3 preferentially innervate the cortex, whereas those co-expressing thyrotropin-releasing hormone innervate subcortical regions in particular the hypothalamus. Reconstruction of 50 individual DR serotonin neurons revealed diverse and segregated axonal projection patterns at the single-cell level. Together, these results provide a molecular foundation of the heterogenous serotonin neuronal phenotypes.

The projection targets of a neuronal population are a key feature of its anatomical characteristics. Historically, tissue sectioning, confocal microscopy, and manual scoring of specific regions of interest have been used to generate coarse summaries of mesoscale projectomes. We present here TrailMap, a three-dimensional (3D) convolutional network for extracting axonal projections from intact cleared mouse brains imaged by light-sheet microscopy. TrailMap allows region-based quantification of total axon content in large and complex 3D structures after registration to a standard reference atlas. The identification of axonal structures as thin as one voxel benefits from data augmentation but also requires a loss function that tolerates errors in annotation. A network trained with volumes of serotonergic axons in all major brain regions can be generalized to map and quantify axons from thalamocortical, deep cerebellar, and cortical projection neurons, validating transfer learning as a tool to adapt the model to novel categories of axonal morphology. Speed of training, ease of use, and accuracy improve over existing tools without a need for specialized computing hardware. Given the recent emphasis on genetically and functionally defining cell types in neural circuit analysis, TrailMap will facilitate automated extraction and quantification of axons from these specific cell types at the scale of the entire mouse brain, an essential component of deciphering their connectivity.