There is an ongoing debate regarding the nature of memory deficits that occur in the early stages of mild cognitive impairment (MCI). MCI has been associated with atrophy to regions that process objects, namely perirhinal and lateral entorhinal cortices. However, it is currently unclear whether older adults with early MCI will show memory deficits that are specific to objects, or whether they will also show memory deficits for other stimulus classes, such as scenes. We tested 75 older adults using an object and scene recognition task with stimulus-specific interference (i.e., exposure to irrelevant object or scene stimuli). We found an interaction (P = 0.05) whereby scores on the Montreal Cognitive Assessment, a neuropsychological test with high sensitivity to MCI, shared a stronger relationship with object recognition than with scene recognition performance. Interestingly, this relationship was not modulated by the stimulus category of interfering items. To further explore these findings, we also tested an amnesic patient (DA) with known medial temporal lobe damage. Like older adults with early signs of MCI, DA showed poorer object recognition than scene recognition performance. Additionally, his performance was not modulated by the stimulus category of interfering material. By demonstrating that object memory is more predictive of cognitive decline than scene memory, these findings support the notion of perirhinal and lateral entorhinal cortex dysfunction in the early stages of MCI. © 2016 Wiley Periodicals, Inc.