Objective: The aim of this study was to identify the factors associated with differences between how Alzheimer’s disease (AD) patients and their caregivers rate the patient’s health-related quality of life (QoL). Methods: Cross-sectional, multicentre study. Patients were 65 years or more, suffering from mild to moderate AD, native French speakers, with a main caregiver. Interrater agreement of the QoL-AD was assessed using the intraclass coefficient. A generalised linear model was used to identify factors related to the difference in health-related QoL scores between patients and their caregivers. Results: The 122 patients of the study were 82 ± 6 years old and mainly women (69%). Independent factors related to the difference between patients and caregivers were: Mini Mental State Exam score (β = 0.32; 95% CI = 0.05–0.59); instrumental activities of daily living score (β = –0.61; 95% CI = –1.14 to –0.07); total Neuropsychiatric Inventory score (β = 0.10; 95% CI = 0.05–0.59), and Zarit’s burden score (β = 0.09; 95% CI = 0.01–0.17). Conclusion: Practitioners must take into account the trend towards underestimation when health-related QoL is rated by caregivers or proxies.