Background: The impact of occupational exposure to solvents on cognitive ageing remains unclear. We examined whether long-term occupational exposure is associated with poor cognitive performance in late midlife. Methods: Participants in the GAZEL cohort, set up in 1989, are employees of the French national electricity and gas company. Data on the working environment were used to create measures of cumulative exposures to solvents using a job-exposure matrix. In 2002–2004, cognitive performance was assessed using the Digit Symbol Substitution Test (DSST) and the Mini Mental State Examination in 5,242 participants (aged 55–65 years). Results: In cross-sectional analysis using multiple logistic regression, there was a greater risk of poor cognitive performance (DSST score <25th percentile) among those with high exposure to benzene (OR = 1.58; 95% CI 1.31–1.90) and the grouped categories of chlorinated (OR = 1.39; 95% CI 1.3–2.3), aromatic (OR = 1.76; 95% CI 1.08–2.87), and petroleum solvents (OR = 1.50; 95% CI 1.23–1.81). Conclusions: These results suggest that occupational exposures to solvents may be associated later in life with cognitive impairment, even after taking into account the effects of education, employment grade, and numerous health factors.