As a new type of high-performance material, gradient structural steel is widely used in engineering fields due to its unique microstructure and excellent mechanical properties. For the prevalent fatigue failure problem, the rate of change of the local grain size gradients along the structure (referred to as the gradient rate) is a key parameter in the design of gradient structures, which significantly affects the fatigue performance of gradient structural steel. In this work, a method of ‘Voronoi primary + secondary modeling’ is adopted to successfully establish three typical high-strength steel models corresponding to the convex, linear, and concave type gradient rates for gradient structures, focusing on the stress-strain response and crack propagation in structural steel with different gradient rates under cyclic loading. It is found that the concave gradient rate structural model is dominated by finer grains with larger volume fraction, which is conducive to hindering fatigue crack propagation and has the longest fatigue life; the linear structure is the second one, and the convex structure is the shortest one. The simulation results in this work are consistent with the relevant experimental phenomena. Therefore, when regulating the gradient rate, priority should be given to increasing the volume fraction of fine grains and designing a gradient rate structure dominated by fine grains to improve the fatigue life of the material.