The complex biological mechanisms underlying human brain aging remain incompletely understood. To investigate this, we utilized multimodal magnetic resonance imaging and artificial intelligence (AI) to examine the genetic heterogeneity of the brain age gap (BAG) derived from gray matter volume (GM-BAG), white matter tract (WM-BAG), and functional connectivity (FC-BAG). Sixteen significant genomic loci were identified, with GM-BAG loci showing abundant associations with neurodegenerative and neuropsychiatric traits, WM-BAG for cancer and Alzheimer's disease (AD), and FC-BAG for only insomnia. The gene-drug-disease network further corroborated these associations by highlighting genes linked to GM-BAG for the treatment of neurodegenerative and neuropsychiatric disorders, and WM-BAG genes for cancer therapy. GM-BAG showed the highest enrichment of heritability in conserved regions, while in WM-BAG, the 5' untranslated regions exhibited the highest heritability enrichment; oligodendrocytes and astrocytes showed significant heritability enrichment in WM and FC-BAG, respectively. Notably, Mendelian randomization identified risk causal effects of triglyceride-to-lipid ratio in VLDL and type 2 diabetes on GM-BAG, and AD on WM-BAG. These findings suggest that interventions targeting these factors and diseases may ameliorate human brain health. Overall, our results provide valuable insights into the genetic heterogeneity of human brain aging, with potential implications for lifestyle and therapeutic interventions.