Alzheimer's disease (AD) progresses insidiously from the preclinical stage to dementia. While people with subjective cognitive decline (SCD) have normal cognitive performance, some may be in the preclinical stage of AD. Neurofibrillary tangles appear first in the transentorhinal cortex, followed by the entorhinal cortex in the clinically silent stage of AD. We expected the earliest changes in subjects with SCD to occur in medial temporal subfields other than the hippocampal proper. These selective structural changes would affect specific memory subcomponents. We used the Family Picture subtest of the Wechsler Memory Scale-III, which was modified to separately compute character, activity, and location subscores for episodic memory subcomponents. We recruited 43 subjects with SCD, 44 subjects with amnesic mild cognitive impairment, and 34 normal controls. MRI was used to assess cortical thickness, subcortical gray matter volume, and fractional anisotropy. The results demonstrated that SCD subjects showed significant cortical atrophy in their bilateral parahippocampus and perirhinal and the left entorhinal cortices but not in their hippocampal regions. SCD subjects also exhibited significantly decreased mean fractional anisotropy in their bilateral uncinate fasciculi. The diminution of cortical thickness over the mesial temporal subfields corresponded to brain areas with early tangle deposition, and early degradation of the uncinate fasciculus was in accordance with the retrogenesis hypothesis. The parahippocampus and perirhinal cortex contribute mainly to context association memory while the entorhinal cortex, along with the uncinate fasciculus, contributes to content-related contextual memory. We proposed that context association and related memory structures are vulnerable in the SCD stage.