In situ real-time electron microscope observations of metals irradiated with ultrahigh-energy electrons or energetic ions show that the dynamics of microstructural evolution in these materials is strongly influenced by long-range elastic interactions between mobile nanoscale radiation defects. Treating long-range interactions is also necessary for modeling microstructures formed in ex situ high-dose-rate ion-beam irradiation experiments, and for interpolating the ion-beam irradiation data to the low-dose-rate limit characterizing the neutron irradiation environments of fission or fusion power plants. We show that simulations, performed using an algorithm where nanoscale radiation defects are treated as interacting Langevin particles, are able to match and explain the real-time dynamics of nanodefects observed in in situ electron microscope experiments.