Aging coincides with a decline in map learning ability, but it is unclear to what extent different aspects of the mental representation are susceptible. The present study aimed to investigate knowledge about landmarks, their positions and distances (categorical and distance relations, respectively) in relation to aging as well as cognitive functioning (measured with the Montreal Cognitive Assessment [MoCA]), visuospatial abilities, and self-reported wayfinding inclinations. Thirty young adults and 60 older adults (30 aged 63–74 and 30 aged 75–86) learned a map, freely recalled the landmarks and performed a map drawing task (considering the number of landmarks missing, position accuracy and distance accuracy). Before that, older participants were also assessed regarding their general cognitive functioning (MoCA) and a series of visuospatial measures. The results show age-related differences among adults in recalling landmarks and in both categorical and distance relations, with a worsening of performance of old-olds only in the former. Older adults’ MoCA score related to accuracy in the three measures, and an additional role of spatial anxiety was found for distance accuracy. Above and beyond the age-related decline, the quality of older people’s spatial mental representation is related to higher general cognitive level and lower spatial anxiety.