Academic fields exhibit substantial levels of gender segregation. Here, we investigated differences in field-specific ability beliefs (FABs) as an explanation for this phenomenon. FABs may contribute to gender segregation to the extent that they portray success as depending on “brilliance” (i.e., exceptional intellectual ability), which is a trait culturally associated with men more than women. Although prior work has documented a relation between academic fields’ FABs and their gender composition, it is still unclear what the underlying dynamics are that give rise to gender imbalances across academia as a function of FABs. To provide insight into this issue, we custom-built a new data set by combining information from the author-tracking service Open Researcher and Contributor ID (ORCID) with information from a survey of U.S. academics across 30 fields. Using this expansive longitudinal data set (Ns = 86,879–364,355), we found that women were underrepresented among those who enter fields with brilliance-oriented FABs and overrepresented among those who exit these fields. We also found that FABs’ association with women’s transitions across academic fields was substantially stronger than their association with men’s transitions. With respect to mechanisms, FABs’ association with gender segregation was partially explained by the fact that women encounter more prejudice in fields with brilliance-oriented FABs. With its focus on the dynamic patterns shaping segregation and its broad scope in terms of geography, career stage, and historical time, this research makes an important contribution toward understanding the factors driving gender segregation in academia.