Rock size has a significant impact on the mechanical behavior and deformation performance of soil-rock mixture. This paper aims at investigating and isolating the effect of rock size on the soil-rock mixture in mechanical behavior and deformation behavior and making clear how this factor influences them. Taking the material heterogeneity into account, a two-dimensional meso-scale numerical method for modeling soil-rock mixture is presented, which considers the rocks and soil matrix as separate constituents. Based on the comparison of stress-strain relationships between the numerical simulations and laboratory tests, it is validated that the meso-scale model could describe the macroscopic mechanical behavior of soil-rock mixture well. Then, using the meso-scale method, 160 groups of simulations on soil-rock mixture with different rock sizes under different rock contents are carried out. The shear strength and deformation performance of soil-rock mixture are explored, and their variation laws with rock sizes are discussed subsequently. The simulation results indicate that the soil-rock mixture samples with smaller rocks exhibit a higher peak shear strength, which shows a linear changing trend with rock size. There are no significant changes that took place in the initial elastic modulus under different rock sizes. However, the shear failure modes will present three types as tympany, necking, and cross shear band with the change of rock size. Finally, the influence mechanism of rock size on SRM shear strength and deformation performance is analyzed theoretically.