What can be expected in normal aging, and where does normal aging stop and pathological neurodegeneration begin? With the slow progression of age-related dementias such as Alzheimer’s Disease (AD), it is difficult to distinguish age-related changes from effects of undetected disease. We review recent research on changes of the cerebral cortex and the hippocampus in aging and the borders between normal aging and AD. We argue that prominent cortical reductions are evident in fronto-temporal regions in elderly even with low probability of AD, including regions overlapping the default mode network. Importantly, these regions show high levels of amyloid deposition in AD, and are both structurally and functionally vulnerable early in the disease. This normalcy-pathology homology is critical to understand, since aging itself is the major risk factor for sporadic AD. Thus, rather than necessarily reflecting early signs of disease, these changes may be part of normal aging, and may inform on why the aging brain is so much more susceptible to AD than is the younger brain. We suggest that regions characterized by a high degree of life-long plasticity are vulnerable to detrimental effects of normal aging, and that this age-vulnerability renders them more susceptible to additional, pathological AD-related changes. We conclude that it will be difficult to understand AD without understanding why it preferably affects older brains, and that we need a model that accounts for age-related changes in AD-vulnerable regions independently of AD-pathology.