This article investigates longitudinal imaging characteristics of early cognitive decline during normal aging, leveraging on high-dimensional imaging pattern classification methods for the development of early biomarkers of cognitive decline. By combining magnetic resonance imaging (MRI) and resting positron emission tomography (PET) cerebral blood flow (CBF) images, an individualized score is generated using high-dimensional pattern classification, which predicts subsequent cognitive decline in cognitively normal older adults of the Baltimore Longitudinal Study of Aging. The resulting score, termed SPARE-CD (Spatial Pattern of Abnormality for Recognition of Early Cognitive Decline), analyzed longitudinally for 143 cognitively normal subjects over 8 years, shows functional and structural changes well before (2.3–2.9 years) changes in neurocognitive testing (California Verbal Learning Test [CVLT] scores) can be measured. Additionally, this score is found to be correlated to the [11C] Pittsburgh compound B (PiB) PET mean distribution volume ratio at a later time. This work indicates that MRI and PET images, combined with advanced pattern recognition methods, may be useful for very early detection of cognitive decline.