Older adults consistently show elevated rates of false recognition of new items that are related to studied items. This finding has been largely attributed to a greater tendency for older adults to rely on conceptual gist during memory recognition tasks. However, perceptual factors may also be implicated considering that related items are not only conceptually but also perceptually similar. While some findings do suggest that age-related increases in false recognitions can be driven by perceptual factors, little is known about the nature and circumstances under which these factors operate. To address this gap, we measured basic visual ability as well as false recognition for four different image categories (upright faces, inverted faces, chairs, houses) in younger ( n = 34) and older ( n = 34) adults. Each image category represented different levels of variability in perceptual similarity and pre-experimental exposure. Perceptual similarity was objectively defined on the basis of the low-level properties of the images. We found evidence that perceptual similarity can contribute to elevated rates of false recognition in older adults. Our results also suggest that declines in basic visual abilities influence elevated false recognition in older adults for perceptually similar but not perceptually dissimilar items. We conclude that both perceptual and conceptual similarity can drive age-related differences in false recognition.