Alexithymia is a personality trait characterised by difficulties identifying feelings (DIF), describing feelings (DDF), and externally oriented thinking (EOT). Alexithymia has been associated with poorer memory, at least for emotive materials, and recently, with executive and neural dysfunction. Aging is also accompanied by poorer memory and executive functioning (EF), neural dysfunction, and increasing alexithymia. Thus, the hypothesis of a general cognitive impairment in alexithymia, particularly in elders, needs investigation. Three large, independent, cross-sectional experiments (n = 296, 139 and 121, respectively) investigated memory and EF in healthy adults, ranging from young to old adulthood, with age, sex, and the three Toronto Alexithymia Scale-20 subscales (DIF, DDF, EOT) as predictors in hierarchical regressions. Across studies, alexithymia contributed to poorer memory (via EOT) and EF (via DIF), in younger and older adults. Additionally, these effects occurred in non-emotive contexts with neutral stimuli. Moreover, although memory was worse with greater age and poor EF contributed to poor memory, those who had both high EOT and poor EF had particularly poor memory. Thus, alexithymia (particularly via high DIF or high EOT) is a risk factor for age-related cognitive decline. Further research should clarify the direction and nature of these complex relationships.