Background: Educational attainment and verbal intelligence, which indirectly reflect an individual's cognitive reserve (CR), is suggested to buffer the effect of late-life brain degradation on cognitive performance outcome. We aimed to explore how the relationship between whole grey matter volume (GMV) and episodic memory function is altered by CR proxy as well as age in healthy older adults. Methods: Elderly Verbal Learning Test (EVLT) and structural magnetic resonance imaging were administered to 110 community-residing older adults. Moderated moderation model tested whether the association between whole GMV and episodic memory was moderated by both CR and chronological age. Results: The results showed that the moderating effect of CR on Immediate Recall, Short-delay Recall, and Recognition scores of EVLT differed across age groups. The elderly with higher CR showed steeper GMV effect on EVLT at the Age-Younger condition, while such moderating effect was reversed in the Age-Older condition, suggesting an alleviated brain atrophy effect in higher CR elderly. Conclusion: These findings suggest that although higher CR elderly may exhibit earlier GMV-related memory decline, the buffering effect of CR on the cognitive decline due to brain atrophy would become more evident in old-old elderly people who are likely to have accumulated more neuropathological changes. This study underscores chronological age as an important moderating factor in examining the moderating role of CR in late-life memory function.