Accumulating research has linked Mediterranean diet (MeDi) adherence with slower cognitive decline and reduced Alzheimer’s disease (AD) risk. However, no study to-date has examined the relationship between MeDi adherence and accumulation of cerebral Aβ-amyloid (Aβ; a pathological hallmark of AD) in older adults. Cognitively normal healthy control participants of the Australian Imaging, Biomarkers and Lifestyle (AIBL) Study of Ageing completed the Cancer Council of Victoria Food Frequency Questionnaire at baseline, which was used to construct a MeDi score for each participant (score range 0–9; higher score indicating higher adherence). Cerebral Aβ load was quantified by Pittsburgh Compound B positron emission tomography at baseline, 18 and 36 months: Only individuals categorised as “Aβ accumulators”, and thus considered to be on the AD pathway, were included in the analysis (N = 77). The relationship between MeDi adherence, MeDi components, and change in cerebral Aβ load (baseline to 36 months) was evaluated using Generalised Linear Modelling, accounting for age, gender, education, Apolipoprotein E ε4 allele status, body mass index and total energy intake. Higher MeDi score was associated with less Aβ accumulation in our cohort (β = −0.01 ± 0.004, p = 0.0070). Of the individual MeDi score components, a high intake of fruit was associated with less accumulation of Aβ (β = −0.04 ± 0.01, p = 0.00036). Our results suggest MeDi adherence is associated with reduced cerebral AD pathology accumulation over time. When our results are considered collectively with previous data linking the MeDi to slower cognitive decline, it is apparent that MeDi adherence warrants further investigation in the quest to delay AD onset.