Intracrystalline chemical diffusion offers valuable insights into the durations of metamorphic and igneous processes. However, it can yield timescale estimates for orogenic events that are considerably shorter than those obtained via isotopic geochronology. One potential explanation for the discrepancy is that diffusion of species with different molar volumes may generate intracrystalline, compositional stresses that limit diffusional relaxation. Here, we show that compositional stress can increase the interpreted timescales of lithospheric processes by an order of magnitude or more by investigating a well-characterized garnet from the Barrovian sillimanite zone, Scotland. The garnet records sharp micrometer-scale variations in calcium and iron contents that standard diffusion treatments predict should relax in 1–10 kyr during peak thermal metamorphism. Conversely, compositional stress allows these features to persist for >100 kyr timescales, consistent with independent estimates from strontium-in-apatite tracer diffusion chronometry. Therefore, accounting for compositional stress can dramatically alter the interpretation of fundamental lithospheric processes.