Grazing exclusion (GE) and rest grazing (RG) are important management systems for the restoration of degraded grassland ecosystems. In order to evaluate the effects and mechanisms of different grazing management systems on the scale dependency of species diversity, plant community indices and soil variables were determined in 32 plots in the Qilian Mountains in Gansu Province. The results show that diversity effects and their regulating mechanisms had space-scale dependence under different grazing management systems. The species richness and species diversity indices of RG grassland were significantly higher than those of GE at the regional scale. Species richness for RG and GE plots was 128 and 98, respectively, and the Shannon–Wiener and Simpson indices for RG and GE plots were 2.26 and 2.16, and 0.85 and 0.83, respectively. Additionally, three grazing management systems had a significant effect on species richness in mountain meadows, but different management systems had significantly different effects on species diversity indices in mountain meadows and temperate steppes. Meanwhile, soil variables only influenced species diversity at the regional scale. Most of the plant and soil variables at each scale had positive effects on species diversity. However, a negative correlation was seen between biodiversity and species coverage, mean plant height, soil porosity (SP) and bulk density (BD) under the two contrasting grazing management systems. In conclusion, choosing RG at the regional scale and selecting grazing management systems according to different grassland types at the local scale can help to restore degraded grassland vegetation.