It is important to understand the molecular determinants for microstructures of human brain. However, past genome-wide association studies (GWAS) on microstructures of human brain have had limited results due to methodological constraints. Here, we adopt advanced imaging processing methods and multivariate GWAS on two large scale imaging genetic datasets (UK Biobank and Adolescent Brain Cognitive Development study) to identify and validate key genetic association signals. We discovered 503 unique genetic loci that explained more than 50% of the average heritability across imaging features sensitive to tissue compartments. The genome-wide signals are strongly overlapped with neuropsychiatric diseases, cognitive functions, risk tolerance, and immune responses. Our results implicate the shared molecular mechanisms between tissue microstructures of brain and neuropsychiatric outcomes with astrocyte involvement in the early developmental stage.