Objectives: Temporal lobe epilepsy (TLE) is commonly associated with mesiotemporal pathology and widespread alterations of grey and white matter structures. Evidence supports a progressive condition although the temporal evolution of TLE is poorly defined. This ENIGMA-Epilepsy study utilized multimodal magnetic resonance imaging (MRI) data to investigate structural alterations in TLE patients across the adult lifespan. We charted both grey and white matter changes and explored the covariance of age-related alterations in both compartments. Methods: We studied 769 TLE patients and 885 healthy controls across an age range of 17-73 years, from multiple international sites. To assess potentially non-linear lifespan changes in TLE, we harmonized data and combined median split assessments with cross-sectional sliding window analyses of grey and white matter age-related changes. Covariance analyses examined the coupling of grey and white matter lifespan curves. Results: In TLE, age was associated with a robust grey matter thickness/volume decline across a broad cortico-subcortical territory, extending beyond the mesiotemporal disease epicentre. White matter changes were also widespread across multiple tracts with peak effects in temporo-limbic fibers. While changes spanned the adult time window, changes accelerated in cortical thickness, subcortical volume, and fractional anisotropy (all decreased), and mean diffusivity (increased) after age 55 years. Covariance analyses revealed strong limbic associations between white matter tracts and subcortical structures with cortical regions. Conclusions: This study highlights the profound impact of TLE on lifespan changes in grey and white matter structures, with an acceleration of aging-related processes in later decades of life. Our findings motivate future longitudinal studies across the lifespan and emphasize the importance of prompt diagnosis as well as intervention in patients.