Cognitive decline and carotid artery atheroma are common at older ages. In community-dwelling subjects, we assessed cognition at ages 70, 73 and 76 and carotid Doppler ultrasound at age 73, to determine whether carotid stenosis was related to cognitive decline. We used latent growth curve models to examine associations between four carotid measures (internal carotid artery stenosis, velocity, pulsatility and resistivity indices) and four cognitive ability domains (memory, visuospatial function, crystallised intelligence, processing speed) adjusted for cognitive ability at age 11, current age, gender and vascular risk factors. Amongst 866 participants, carotid stenosis (median 12.96%) was not associated with cognitive abilities at age 70 or cognitive decline from age 70 to 76. Increased ICA pulsatility and resistivity indices were associated with slower processing speed (both P < 0.001) and worse visuospatial function (P = 0.036, 0.031, respectively) at age 70, and declining crystallised intelligence from ages 70 to 76 (P = 0.008, 0.006, respectively). The findings suggest that vascular stiffening, rather than carotid luminal narrowing, adversely influences cognitive ageing and provides a potential target for ameliorating age-related cognitive decline.