Utilizing data from the Million Veteran Program (MVP), we investigated the genetic determinants underlying total testosterone levels via a multi-ancestral analysis of 124,593 individuals of European (n=88,385), African (n=25,235) and Hispanic (n=10,973) ancestry. We identified 46 trans-ancestry variants and 17 ancestry-specific variants, of which 14 trans-ancestry variants and 15 ancestry-specific variants are novel associations with testosterone. Results implicate genes regulating testosterone shared across ancestral groups, which include SHBG, JMJD1C, FXR2, SENP3, TNFSF12-TNFSF13 while also implicating genes such as MSN, DMD, VSIG4, CHEK2, TKTL1 that may underlie ancestry-group differences in testosterone regulation. We also linked testosterone variants on the X chromosome with differential risk of chronic kidney disease and hereditary hemolytic anemias in African and Hispanic ancestry groups, respectively. Lastly, we constructed a polygenic score from our 46 trans-ancestry variants and associated it with testicular dysfunction, hyperlipidemia, gout and prostate cancer with stronger prostate cancer associations in Hispanic and African ancestry groups compared to the European ancestry group. These findings provide insight into ancestry-specific androgen regulation and identify novel variants for disease risk stratification in patients.