Data-driven fine-grained region discovery in the mouse brain with transformers

Abstract

Technologies such as spatial transcriptomics offer unique opportunities to define the spatial organization of the mouse brain. We developed an unsupervised training scheme and novel transformer-based deep learning architecture to detect spatial domains in mouse whole-brain spatial transcriptomics data. Our model learns local representations of molecular and cellular statistical patterns. These local representations can be clustered to identify spatial domains within the brain from coarse to fine-grained. Discovered domains are spatially regular, even with several hundreds of spatial clusters. They are also consistent with existing anatomical ontologies such as the Allen Mouse Brain Common Coordinate Framework version 3 (CCFv31) and can be visually interpreted at the cell type or transcript level. We demonstrate our method can be used to identify previously uncatalogued subregions, such as in the midbrain, where we uncover gradients of inhibitory neuron complexity and abundance. We apply our method to a separate multi-animal whole-brain spatial transcriptomic dataset and observe that inclusion of both sagittal and coronal tissue slices in region identification improves correspondence of spatial domains to CCF.