The brain-derived neurotropic growth factor (BDNF) gene has been linked to dementia, inflammation, and Apolipoprotein E (APOE) ɛ4 status. We used cerebrospinal fluid (CSF) amyloid-β (Aβ)42 and phosphorylated tau (p-tau) to investigate associations with BDNF polymorphisms and modifications by APOE ɛ4 or inflammation in a memory clinic population (n = 114; subjective cognitive decline, mild cognitive impairment, Alzheimer’s disease). We found distinct pathways to Alzheimer’s disease pathology: Val-Met displayed lower CSF-Aβ 42 in APOE ɛ4+ carriers, independent of p-tau, while Val-Val displayed greater p-tau at higher IL-6 and sub-threshold Aβ 42. This may contribute to resolving some inconsistencies in the BDNF literature and provide possible inroads to specific Aβ and tau interventions depending on BDNF polymorphism.