ObjectivesThis study examined the association between childhood adversity and cognitive impairment in later life and explored the potential moderation effect of gender and race.MethodsThe study sample included 15,133 participants of the Health and Retirement Study (1998–2016 surveys) who had complete data on key study measures and were more than 50. The outcome variable is a dichotomous indicator of cognitive impairment as assessed by the Telephone Interview for Cognitive Status for self-respondents and the 16-item Informant Questionnaire on Cognitive Decline in the Elderly for proxies. A total of six childhood adversity indicators included grade retention, parental substance abuse, physical abuse, trouble with the police, moving due to financial hardship, and receipt of help due to financial hardship in early life. The estimation of the association between childhood adversity and cognitive impairment involved Cox proportional hazards regression. Results: Grade retention had the largest effect on incident cognitive impairment (HR = 1.3, 95% CI = 1.23–1.38, p < 0.001), followed by physical abuse by a parent (HR = 1.10, 95% CI = 1.00–1.20, p = 0.001). The impact of grade retention was more detrimental to women than men (interaction term HR = 0.89, 95% CI = 0.80–1.00, p = 0.048, female as the reference). Parental substance abuse was associated with a lower risk of incident cognitive impairment for most racial groups (HR = 0.89, 95% CI = 0.83–0.95, p = 0.001), but this association was reversed in “non-Hispanic other” race, consisting mainly of Asians (HR = 1.54, 95% CI = 1.05–2.26, p = 0.025).DiscussionSome aspects of childhood adversity continue to harm cognitive functioning in later life, while some events may have the opposite effect, with evidence of heterogeneity across gender and race.