Cerebrovascular reactivity (CVR), is important for determining future risk of cerebrovascular disease. It is unclear if primary aging is associated with reductions in CVR because previous studies often include participants with vascular risk factors. Additionally, the inconsistency in the literature may be due to the inherent difficulty in quantifying intracranial cerebral blood flow and CVR. To address these limitations, we determined the effect of age on CVR in the large intracranial vessels in adults with low vascular risk using state-of-the-art MRI techniques. We also determined if the effect of age on CVR was sex-specific. Young (n = 20; 25 ± 3 years) and older (n = 19; 61 ± 5 years) healthy, physically active adults participated in the study. CVR was measured in response to hypercapnia using 4D flow MRI, which allows for simultaneous angiographic and quantitative blood flow measurements in the intracranial arteries. Older adults had lower global CVR and CVR in multiple intracranial arteries [right and left internal carotid arteries (ICA), right and left middle cerebral arteries (MCA), and basilar artery (BA)] compared with young adults (p < 0.05 for all). In addition, the MCA dilated significantly in response to hypercapnia in young (p < 0.05), but not older adults. Young men demonstrated higher global CVR and CVR in multiple intracranial arteries (ICAs, MCAs, and BA) compared with young women and older men (p < 0.05 for both); however, CVR did not differ between young women and older women. Our results demonstrate that, using 4D flow MRI, primary aging is associated with lower CVR in adults with low vascular risk. In addition, the effect of age on CVR may be driven by men. The 4D flow MRI technique may provide a promising new alternative to measure cerebrovascular physiology without the limitations of commonly used techniques. Future studies could utilize this MRI technique to examine interventions to maintain CVR with advancing age. This study was registered under clinicaltrials.gov # NCT02840851.