Researchers have identified changes in dopaminergic neuromodulation as playing a key role in adult memory decline. Facilitated by technical advancements, recent research has also implicated noradrenergic neuromodulation in shaping late-life memory development. However, it is not yet clear whether these two neuromodulators have distinct roles in age-related cognitive changes. Combining longitudinal high-resolution magnetic resonance imaging of the dopaminergic substantia nigra-ventral tegmental area (SN-VTA) and the noradrenergic locus coeruleus (LC), we found that dopaminergic and noradrenergic integrity are differentially associated with individual differences in younger (n = 69) and older adults' (n = 251) memory performance. While LC integrity was related to better episodic memory across several memory tasks, SN-VTA integrity was linked to working memory. Moreover, consistent with their dense interconnection and a largely shared biosynthesis, dopaminergic and noradrenergic brain regions' integrity were positively related, and correlated with medial temporal lobe volumes. Longitudinally, we found that older age was associated with more-negative change in SN-VTA and LC integrity (time point 1--time point 2; mean delay ~1.9 years). Importantly, changes in LC integrity reliably predicted future episodic memory performance (at time point 3). These findings support the feasibility of in-vivo indices for catecholaminergic integrity with potential clinical utility, given the degeneration of both neuromodulatory systems in several age-associated diseases. Moreover, they highlight differential roles of dopaminergic and noradrenergic neuromodulatory nuclei in late-life cognitive decline.