As the world ages, it becomes urgent to unravel the mechanisms underlying brain aging and find ways of intervening with them. While for decades cognitive aging has been related to localized brain changes, growing attention is now being paid to alterations in distributed brain networks. Functional connectivity magnetic resonance imaging (fcMRI) has become a particularly useful tool to explore large-scale brain networks; yet, the temporal course of connectivity lifetime changes has not been established. Here, an extensive cross-sectional sample (21–85 years old, N = 887) from a public fcMRI database was used to characterize adult lifespan connectivity dynamics within and between seven brain networks: the default mode, salience, dorsal attention, fronto-parietal control, auditory, visual and motor networks. The entire cohort was divided into young (21–40 years, mean ± SD: 25.5 ± 4.8, n = 543); middle-aged (41–60 years, 50.6 ± 5.4, n = 238); and old (61 years and above, 69.0 ± 6.3, n = 106) subgroups. Correlation matrices as well as a mixed model analysis of covariance indicated that within high-order cognitive networks a considerable connectivity decline is already evident by middle adulthood. In contrast, a motor network shows increased connectivity in middle adulthood and a subsequent decline. Additionally, alterations in inter-network interactions are noticeable primarily in the transition between young and middle adulthood. These results provide evidence that aging-related neural changes start early in adult life.