Cognitive aging, especially cognitive control, and processing speed aging have been well-documented in the literature. Most of the evidence was reported based on cross-sectional data, in which inter-individual age effects were shown. However, there have been some studies pointing out the possibility of overlooking intra-individual changes in cognitive aging. To systematically examine whether age-related differences and age-related changes might yield distinctive patterns, this study directly compared cognitive control function and processing speed between different cohorts versus follow-up changes across the adult lifespan. Moreover, considering that cognitive aging has been attributed to brain disconnection in white matter (WM) integrity, this study focused on WM integrity via acquiring diffusion-weighted imaging data with an MRI instrument that are further fitted to a diffusion tensor model (i.e., DTI) to detect water diffusion directionality (i.e., fractional anisotropy, FA; mean diffusivity, MD; radial diffusivity, RD; axial diffusivity, AxD). Following data preprocessing, 114 participants remained for further analyses in which they completed the two follow-up sessions (with a range of 1–2 years) containing a series of neuropsychology instruments and computerized cognitive control tasks. The results show that many significant correlations between age and cognitive control functions originally shown on cross-sectional data no longer exist on the longitudinal data. The current longitudinal data show that MD, RD, and AxD (especially in the association fibers of anterior thalamic radiation) are more strongly correlated to follow-up aging processes, suggesting that axonal/myelin damage is a more robust phenomenon for observing intra-individual aging processes. Moreover, processing speed appears to be the most prominent cognitive function to reflect DTI-related age (cross-sectional) and aging (longitudinal) effects. Finally, converging the results from regression analyses and mediation models, MD, RD, and AxD appear to be the representative DTI measures to reveal age-related changes in processing speed. To conclude, the current results provide new insights to which indicator of WM integrity and which type of cognitive changes are most representative (i.e., potentially to be neuroimaging biomarkers) to reflect intra-individual cognitive aging processes.