Alzheimer’s disease (AD) is a progressive neurodegenerative disease for which no effective treatment exists at present. Previous research has found that exercise reduces the risk of AD. Since the apolipoprotein E (APOE) ε4 allele increases the risk of AD and is associated with faster disease progression than the other isoforms, we aimed to highlight the impact of exercise on AD pathology in APOE ε4 carriers. This review focuses on the effect of exercise on cognitive function, dementia risk, amyloid-β (Aβ) metabolism, lipid metabolism, neuroinflammation, neurotrophic factors and vascularization in APOE ε4 carriers. We searched the literature in the PubMed electronic database using the following search terms: physical activity, exercise, aerobic fitness, training, sport, APOE4, Alzheimer’s disease, AD and dementia. By cross-referencing, additional publications were identified. Selected studies required older adults to take part in an exercise intervention or to make use of self-reported physical activity questionnaires. All included studies were written and published in English between 2000 and 2020. From these studies, we conclude that exercise is a non-pharmacological treatment option for high-risk APOE ε4 carriers to ameliorate the AD pathological processes including reducing Aβ load, protecting against hippocampal atrophy, improving cognitive function, stabilizing cholesterol levels and lowering pro-inflammatory signals. Variation in study design related to age, cognitive outcomes and the type of intervention explained the differences in study outcomes. However, exercise seems to be effective in delaying the onset of AD and may improve the quality of life of AD patients.