Normal aging and Alzheimer's Disease (AD) are accompanied by large-scale alterations in brain organization that undermine brain function. Although hemispheric asymmetry is a global organizing feature of cortex thought to promote brain efficiency, current descriptions of cortical thinning in aging and AD have largely overlooked cortical asymmetry. Consequently, the foundational question of whether and where the cerebral hemispheres change at different rates in aging and AD remains open. First, applying vertex-wise data-driven clustering in a longitudinal discovery sample (aged 20-89; 2577 observations; 1851 longitudinal) we identified cortical regions exhibiting similar age-trajectories of asymmetry across the adult lifespan. Next, we sought replication in 4 independent longitudinal aging cohorts. We show that higher-order regions of cortex that exhibit pronounced asymmetry at age ~20 also show asymmetry change in aging. Results revealed that both leftward and rightward asymmetry is progressively lost on a similar time-scale across adult life. Hence, faster thinning of the (previously) thicker homotopic hemisphere is a feature of aging. This simple organizational principle showed high consistency across multiple aging cohorts in the Lifebrain consortium, and both the topological patterns and temporal dynamics of asymmetry-loss were markedly similar across replicating samples. Finally, we show that regions exhibiting gradual asymmetry-loss over healthy adult life exhibit faster asymmetry-change in AD.Overall, our results suggest a system-wide breakdown in the adaptive asymmetric organization of cortex across adult life which is further accelerated in AD, and may implicate thickness asymmetry as a viable marker for declining hemispheric specialization in aging and AD. The brain becomes progressively disorganized with age, and brain alterations accelerated in Alzheimer's disease may occur gradually over the lifespan. Although hemispheric asymmetry aids efficient network organization, efforts to identify structural markers of age-related decline have largely overlooked cortical asymmetry. Here we show the hemisphere that is thicker when younger, thins faster. This leads to progressive system-wide loss of regional thickness asymmetry across life. In multiple aging cohorts, asymmetry-loss showed high reproducibility topologically across cortex and similar timing-of-change in aging. Asymmetry-change was further accelerated in AD. Our findings uncover a new principle of brain aging -thicker homotopic cortex thins faster -and suggest we may have unveiled a structural marker for a widely-hypothesized decline in hemispheric specialization in aging and AD.