Both sex and gender are characteristics that play a key role in risk and resilience in health and well-being. Current research lacks the ability to quantitatively describe gender and gender diversity, and is limited to endorsement of categorical gender identities, which are contextually and culturally dependent. A more objective, dimensional approach to characterizing gender diversity will enable researchers to advance the health of gender-diverse people by better understanding how genetic factors interact to determine health outcomes. To address this research gap, we leveraged the Gender Self-Report (GSR), a questionnaire that captures multiple dimensions of gender diversity. We then performed polygenic score associations with brain-related traits like cognitive performance, personality, and neuropsychiatric conditions. The GSR was completed by N = 818 independent adults with or without autism in the SPARK cohort, and GSR factor analysis identified two factors: Binary (divergence from gender presumed by designated sex to the opposite) and Nonbinary (divergence from male and female gender norms) Gender Diversity (BGD and NGD, respectively). We performed polygenic associations (controlling for age, sex, and autism diagnostic status) in a subset of N = 452 individuals and found higher polygenic propensity for cognitive performance was associated with greater BGD (β = 0.017, p = 0.049) and NGD (β = 0.036, p = 0.002), and higher polygenic propensity for educational attainment was also associated with greater NGD (β = 0.030, p = 0.015). We did not observe any significant associations with personality or neuropsychiatric polygenic scores in this sample. Overall, our results suggest cognitive processes and gender diversity share overlapping genetic factors, indicating the biological utility of the GSR while also underscoring the importance of quantitatively measuring gender diversity in health research contexts.