Soil erosion is serious in China—the soil in plateau and mountain areas contain a large of rock fragments, and their content and distribution have an important influence on soil erosion. However, there are still no complete results for calculating soil erodibility factor (K) that have corrected rock fragments in China. In this paper, the data available on rock fragments in the soil profile (RFP); rock fragments on the surface of the soil (RFS); and environmental factors such as elevation, terrain relief, slope, vegetation coverage (characterised by normalised difference vegetation index, NDVI), land use, precipitation, temperature, and soil type were used to explore the effects of content of soil rock fragments on calculating of K in China. The correlation analysis, typical sampling area analysis, and redundancy analysis were applied to analyse the effects of content of soil rock fragments on calculating of K and its relationship with environment factors. The results showed that (1) The rock fragments in the soil profile (RFP) increased K. The rock fragments on the surface (RFS) of the soil reduced K. The effect of both RFP and RFS reduced K. (2) The effect of rock fragments on K was most affected by elevation, followed by terrain relief, NDVI, slope, soil type, temperature, and precipitation, but had little correlation with land use. (3) The result of redundancy analysis showed elevation to be the main predominant factor of the effect of rock fragments on K. This study fully considered the effect of rock fragments on calculating of K and carried out a quantitative analysis of the factors affecting the effect of rock fragments on K, so as to provide necessary scientific basis for estimating K and evaluating soil erosion status in China more accurately.