A decline in declarative or explicit memory has been extensively characterized in cognitive aging and is a hallmark of cognitive impairments. However, whether and how implicit perceptual memory varies with aging or cognitive impairment is unclear. Here, we compared implicit perceptual memory and explicit memory measures in three groups of participants: (1) 59 healthy young volunteers (20-30 years); (2) 269 healthy old volunteers (50-90 years) and (3) 21 patients with mild cognitive impairment, i.e., MCI (50-90 years). To measure explicit memory, participants were tested on standard recognition and recall tasks. To measure implicit perceptual memory, we used a classic perceptual priming paradigm. Participants had to report the shape of a visual search pop-out target whose color or position was varied randomly across trials. Perceptual priming was measured as the speedup in response time for targets that repeated in color or position. Our main findings are as follows: (1) Explicit memory was weaker in old compared to young participants, and in MCI patients compared to age-and education-matched controls; (2) Surprisingly, perceptual priming did not always decline with age: color priming was smaller in older participants but position priming was larger; (3) Position priming was less frequent in the MCI group compared to matched controls; (4) Perceptual priming and explicit memory were uncorrelated across participants. Thus, perceptual priming can increase or decrease with age or cognitive impairment, but these changes do not covary with explicit memory.