Retaining walls are structures used to retain earth materials on a slope. Typically, they are designed for static loads, but for highway and railway infrastructures, vehicle-induced dynamic responses are also relevant. Therefore, retaining wall structures are often designed with a factor of safety that is higher than necessary, because it can be challenging to quantify the magnitude of expected dynamic stresses during the design stage. This unnecessary increase in material usage reduces the sustainability of the infrastructures. To improve railway retaining wall sustainability, this paper presents the results from a field monitoring campaign on a heavy-haul rail line with a retaining wall, studying the dynamics induced in response to 30-ton axle load trains running at speeds of between 5 km/h and 100 km/h. The site comprises an earth embankment supported by a gravity retaining wall, with accelerometers on the sleepers, roadbed surface, and retaining wall, velocity sensors on the roadbed, and strain gauges on the rail web to record wheel–rail forces. The vibration intensities collected from various locations are processed to explore the peak particle velocities, maximum transient vibration values, and one-third octave band spectrums. Two transfer functions define the vibration transmission characteristics and attenuation of vibration amplitude along the propagation path. The long-term dynamic stability of the track formation is studied using dynamic shear strain derived from the effective velocity. The peaks of observed contact forces and vibrations are statistically analyzed to assess the impact of train speed on the dynamic behavior of the infrastructure system. Next, a 3D numerical model expresses the maximum stress and displacements on the roadbed surface as a function of train speed. The model evaluates the earth pressures at rest and vehicle-induced additional earth pressures and horizontal wall movement. The investigation provides new insights into the behavior of railway track retaining walls under train loading, and the field data are freely available for other researchers to download. The findings could facilitate the design of more sustainable retaining walls in the future.