Background: Past studies identified a link between weight loss and dementia, but lacked consistent conclusions. We sought to establish this link by examining the weight change profiles before and after dementia diagnosis. Methods: Using data from the Health and Retirement Study (1996–2020), we examined 13,123 participants. We conducted a nested case–control analysis to assess differences in biennial weight change profile while controlling for BMI, longevity polygenic risk scores, and APOE gene variants. Results: Participants with a memory disorder lost weight (−0.63%) biennially, whereas those without a diagnosis did not (+0.013%, p-value < 0.0001). Our case–control study shows a significant difference (p-value < 0.01) in pre-dementia % weight changes between the cases (−0.29%) and controls (0.19%), but not in post-dementia weight changes. The weight loss group have the highest risk (OR = 2.01; p-value < 0.0001) of developing a memory disorder compared to the stable weight and weight gain groups. The observations hold true after adjusting for BMI, longevity polygenic risk scores, and APOE variant in a multivariable model. Conclusions: We observe that weight loss in dementia is a physiological process independent of genetic factors associated with BMI and longevity. Pre-dementia weight loss may be an important prognostic criterion to assess a person’s risk of developing a memory disorder.