To study the mechanical characteristics of ballasted bed under dynamic stability, a three-dimensional discrete element model of ballasted bed with 1000 mm × 700 mm × 550 mm is established based on the discrete element method. Meanwhile, a simulation test bench with the same size is built for simulation analysis and experimental research. The variation law and trend of settlement, lateral resistance, and compactness of ballasted bed under different excitation frequencies are analyzed comprehensively. The results show that when the excitation frequency is constant, the lateral resistance of sleeper, the settlement of sleeper, and the compactness of track bed under sleeper increase first and then tend to be stable. With the increase in horizontal excitation frequency, lateral resistance of ballast bed, sleeper settlement, and compactness first increase and then decrease after stable operation. The values of these parameters are maximized when the excitation frequency is 36 Hz which is the relative optimal horizontal excitation frequency of this model.

Dynamic stabilization operation can effectively improve the mechanical performance of ballast bed by using stabilizer. To improve the stabilization effect, a rigid-flexible coupling dynamics simulation model of the stabilizer-ballast bed system was established according to the actual structure of the stabilizer, and the effect of different operation parameters of stabilizer on the stabilization operation was analyzed. The simulation results showed that: the maximum value of lateral displacement response of the sleeper decreased with the increase in clamping wheel force normal angle, excitation force frequency and downforce; and increased with the increase in excitation force amplitude. The optimized operation parameters were obtained as follows: clamping wheel force normal angle 0°, excitation force frequency 18 Hz, excitation force amplitude 180 kN, downforce 60 kN. A vibration test bench simulating the stabilization operation process was built according to the working principle of the stabilizer, and the reliability of the simulation was verified by testing.