The goal of this study was to identify factors contributing to hippocampal atrophy rate (HAR) in clinically-normal older adults (NC) and participants with mild cognitive impairment (MCI). Longitudinal HAR was measured on T1-weighted MRI, and the contribution of age, gender, ApoE ε4 status, intracranial volume, white matter lesions, and β-amyloid (Aβ) levels to HAR was determined using linear regression. Age-related effects of HAR were compared in Aβ positive (Aβ+) and Aβ negative (Aβ−) participants. Age and Aβ levels had independent effects on HAR in NC, while gender, ApoE ε4 status, and Aβ levels were associated with HAR in MCI. In multivariable models, Aβ levels were associated with HAR in NC; ApoE ε4 and Aβ levels were associated with HAR in MCI. In MCI, age was a stronger predictor of HAR in Aβ− versus Aβ+ participants. HAR was higher in Aβ+ participants, but the majority of HAR was due to factors other than Aβ status. Age-related effects on HAR did not differ between NC versus MCI participants with the same Aβ status. Therefore we conclude that even when accounting for other covariates, Aβ status, and not age, is a significant predictor of HAR; and that the majority of HAR is not accounted for by Aβ status in either NC or MCI.