We analysed the genomic architecture of neuroanatomical diversity using magnetic resonance imaging and SNP data from > 20,000 individuals. Our results replicate previous findings of a strong polygenic architecture of neuroanatomical diversity. SNPs captured from 40% to 54% of the variance in the volume of different brain regions. We observed a large correlation between chromosome length and the amount of phenotypic variance captured, r~0.64 on average, suggesting that at a global scale causal variants are homogeneously distributed across the genome. At a more local scale, SNPs within genes (~51%) captured~1.5-times more genetic variance than the rest; and SNPs with low minor allele frequency (MAF) captured significantly less variance than those with higher MAF: the 40% of SNPs with MAF<5% captured less than one fourth of the genetic variance. We also observed extensive pleiotropy across regions, with an average genetic correlation of rG~0.45. Across regions, genetic correlations were in general similar to phenotypic correlations. By contrast, genetic correlations were larger than phenotypic correlations for the left/right volumes of the same region, and indistinguishable from 1. Additionally, the differences in left/right volumes were not heritable, underlining the role of environmental causes in the variability of brain asymmetry. Our analysis code is available at https://github.com/neuroanatomy/genomic-architecture.