Numerous disordered materials display a monotonous slowing down in their internal dynamics with age. In the case of metallic glasses, this general behavior across different temperatures and alloys has been used to establish an empirical universal superposition principle of time, waiting time, and temperature. Here we demonstrate that the application of a mechanical stress within the elastic regime breaks this universality. Using in-situ x-ray photon correlation spectroscopy (XPCS) experiments, we show that strong fluctuations between slow and fast structural dynamics exist, and that these generally exhibit larger relaxation times than in the unstressed case. On average, relaxation times increase with stress magnitude, and even preloading times of several days do not exhaust the structural dynamics under load. A model Lennard-Jones glass under shear deformation replicates many of the features revealed with XPCS, indicating that local and heterogeneous microplastic events can cause the strongly non-monotonous spectrum of relaxation times.