Purpose: To investigate predictors of cognitive decline after whole brain radiotherapy (WBRT) for brain metastases. Methods: A secondary analysis of a phase 2 clinical trial was conducted in patients who received stereotactic radiosurgery for 1-10 brain metastases and WBRT (NCT01046123). The Montreal Cognitive Assessment (MoCA) was performed at baseline and every three months after WBRT. Baseline T2-weighted fluid attenuation inversion recovery magnetic resonance imaging was independently assessed by two neuroradiologists for the presence of white matter hyperintensities (WMH) using the Fazekas visual rating scale. WMH were also manually segmented for volumetric analysis. Univariable and multivariable logistic regression were used to test the association between baseline variables and MoCA score decline. Results: Forty-six patients survived ≥3 months after treatment. Age (OR 1.12 (1.04 – 1.21, p < 0.01), baseline WMH volume (OR 1.20, 95% CI 1.06 – 1.52, p = 0.02) and baseline Fazekas score ≥ 3/6 (OR 6.4, 95% CI 1.7 – 24.7, p < 0.01) were predictive of MoCA score decline. In multivariable analysis, age was the only significant predictor of MoCA decline. However, all three patients with pretreatment leukoencephalopathy (Fazekas score = 6/6) had notable adverse outcomes due to cognitive impairment: one required full-time home nursing support and two were institutionalized. Conclusion: A greater decline in cognition after WBRT was observed in older patients and patients with a higher baseline WMH burden. Although this study is small, we propose that a Fazekas score of 6 should be a relative contraindication to WBRT. Trial Registration: clinicaltrials.gov identifier NCT01046123. First posted January 11, 2010. https://clinicaltrials.gov/ct2/show/NCT01046123