Early adversity and socioeconomic disadvantage are risk factors associated with diminished cognitive outcomes during development. Recent studies also provide evidence that upbringings characterized by stressful experiences and markers of disadvantage during childhood, such as lower parental education or household income, are associated with variation in brain structure. Although disadvantage often confers adversity, these are distinct risk factors whose differential influences on neurodevelopment and neurocognitive outcomes are not well characterized. We examined pathways linking parental education, adverse experiences, brain structure, and cognitive performances through an analysis of 1,413 typically-developing youth, ages 8 through 21, in the Philadelphia Neurodevelopmental Cohort. Parental education and adverse experiences had unique associations with cortical surface area and subcortical volume as well as cognitive performance across several domains. Associations between parental education and several cognitive tasks were explained, in part, by variation in cortical surface area. In contrast, associations between adversity and cognitive tasks were explained primarily by variation in subcortical volume. A composite neurodevelopmental factor derived from principal component analysis of cortical thickness, cortical surface area, and subcortical volume mediated independent associations between both parental education and adverse experiences with reading, geometric reasoning, verbal reasoning, attention, and emotional differentiation tasks. Our analysis provides novel evidence that socioeconomic disadvantage and adversity influence neurodevelopmental pathways associated with cognitive outcomes through independent mechanisms.