The brainstem contains numerous nuclei and tracts with vital functions. Genome-wide associations with brainstem substructure volumes are explored in European individuals, however other ancestries are under-represented. Here, we conducted the cross-ancestry genome-wide association meta-analyses in 72,717 individuals for brainstem and 48,522 for eight substructure volumes, including 7,096 Chinese Han individuals. We identified 122 genetic loci associated with brainstem and substructure volumes at P < 5.56 ×10− 9, including 46 new loci. Three associations had different effect sizes and 292 associations had similar effect sizes between ancestries. We prioritized 550 genes for these brainstem volumetric traits, primarily enriching for neural development. We identified the shared and distinct genetic loci, genes, and pathways for midbrain, pons, and medulla volumes, and the shared genetic architectures with brainstem-related neuropsychiatric disorders and physiological functions. The results provide new insight into genetic architectures of brainstem and substructure volumes and their genetic associations with brainstem physiologies and pathologies.