The surface integrity of structural materials after surface mechanical strengthening has a significant effect on service performance. Herein, the 50CrMnMoVNb spring steels with different microstructures and strengths are surface strengthened by shot peening (SP) with the same parameter, and the effect of matrix strength on surface integrity, including the roughness, residual stress, and gradient structure, is systematically studied. The experimental results show that both the line roughness and surface roughness decrease with the increase in matrix strength; the microhardness enhancement at the topmost surface layer and thickness of gradient layer decrease with the increase in matrix strength, whereas the surface residual stress and the maximum compressive residual stress display an increasing trend with the increase in matrix strength, and both the depth at the maximum compressive residual stress and the thickness of compressive residual stress zone are stable with a value of ≈75 and ≈300 μm, respectively. Inspired by these results and discussions, the surface integrity of the shot peened (SPed) metallic materials would be improved greatly by composition adjustment and microstructure optimization to regulate the matrix strength, as well as the SP process optimization.