ImportanceCognitive impairment in depression is poorly understood. Family history of depression is a potentially useful risk marker for cognitive impairment, facilitating early identification and targeted intervention in those at highest risk, even if they do not themselves have depression. Several research cohorts have emerged recently that enable findings to be compared according to varying depths of family history phenotyping, in some cases also with genetic data, across the life span.ObjectiveTo investigate associations between familial risk of depression and cognitive performance in 4 independent cohorts with varied depth of assessment, using both family history and genetic risk measures.Design, Setting, and ParticipantsThis study used data from the Three Generations at High and Low Risk of Depression Followed Longitudinally (TGS) family study (data collected from 1982 to 2015) and 3 large population cohorts, including the Adolescent Brain Cognitive Development (ABCD) study (data collected from 2016 to 2021), National Longitudinal Study of Adolescent to Adult Health (Add Health; data collected from 1994 to 2018), and UK Biobank (data collected from 2006 to 2022). Children and adults with or without familial risk of depression were included. Cross-sectional analyses were conducted from March to June 2022.ExposuresFamily history (across 1 or 2 prior generations) and polygenic risk of depression.Main Outcomes and MeasuresNeurocognitive tests at follow-up. Regression models were adjusted for confounders and corrected for multiple comparisons.ResultsA total of 57 308 participants were studied, including 87 from TGS (42 [48%] female; mean [SD] age, 19.7 [6.6] years), 10 258 from ABCD (4899 [48%] female; mean [SD] age, 12.0 [0.7] years), 1064 from Add Health (584 [49%] female; mean [SD] age, 37.8 [1.9] years), and 45 899 from UK Biobank (23 605 [51%] female; mean [SD] age, 64.0 [7.7] years). In the younger cohorts (TGS, ABCD, and Add Health), family history of depression was primarily associated with lower performance in the memory domain, and there were indications that this may be partly associated with educational and socioeconomic factors. In the older UK Biobank cohort, there were associations with processing speed, attention, and executive function, with little evidence of education or socioeconomic influences. These associations were evident even in participants who had never been depressed themselves. Effect sizes between familial risk of depression and neurocognitive test performance were largest in TGS; the largest standardized mean differences in primary analyses were −0.55 (95% CI, −1.49 to 0.38) in TGS, −0.09 (95% CI, −0.15 to −0.03) in ABCD, −0.16 (95% CI, −0.31 to −0.01) in Add Health, and −0.10 (95% CI, −0.13 to −0.06) in UK Biobank. Results were generally similar in the polygenic risk score analyses. In UK Biobank, several tasks showed statistically significant associations in the polygenic risk score analysis that were not evident in the family history models.Conclusions and RelevanceIn this study, whether assessed by family history or genetic data, depression in prior generations was associated with lower cognitive performance in offspring. There are opportunities to generate hypotheses about how this arises through genetic and environmental determinants, moderators of brain development and brain aging, and potentially modifiable social and lifestyle factors across the life span.