Advanced age and the APOE ε4 allele are the two biggest risk factors for Alzheimer’s disease (AD) and declining cognitive function. We describe a universal gauge to measure molecular brain age using transcriptome analysis of four human postmortem cohorts (n = 673, ages 25–97) free of neurological disease. In a fifth cohort of older subjects with or without neurological disease (n = 438, ages 67–108), we show that subjects with brains deviating in the older direction from what would be expected based on chronological age show an increase in AD, Parkinson’s disease, and cognitive decline. Strikingly, a younger molecular age (−5 yr than chronological age) protects against AD even in the presence of APOE ε4. An established DNA methylation gauge for age correlates well with the transcriptome gauge for determination of molecular age and assigning deviations from the expected. Our results suggest that rapid brain aging and APOE ε4 are synergistic risk factors, and interventions that slow aging may substantially reduce risk of neurological disease and decline even in the presence of APOE ε4.