Grasslands support a rich diversity that plays a pivotal role in ecological balance, carbon sequestration, and climate regulation. Grassland ecosystems are facing a loss of biodiversity and a decline in soil function due to precipitation variations caused by climate change. However, variations in grassland species diversity are not clearly known, especially under the combined effects of precipitation and grazing. Based on a 5‐year split‐plot experiment with two grazing regimes (i.e., moderate grazing and grazing prohibition) along with three precipitation controls (ambient precipitation and ±50% precipitation) in Inner Mongolian grasslands, we researched the response of precipitation variability and grazing prohibition to vegetation diversity indices and investigated the response of vegetation community composition and biomass to these factors. Five years results showed that increasing precipitation resulted in a significant increase in the coverage of perennial forbs, annual and biennial plants, and Gramineae. Grazing prohibition and increased precipitation significantly increased biomass and species richness compared to all other treatment combinations. Interestingly, grazing exclusion significantly reduced the percentage of Amaryllidaceae, especially in the presence of altered precipitation. The vegetation community composition of a typical grassland was correlated with soil volumetric water content (VWC), bulk density (SBD) and pH, soil organic carbon (SOC), and total nitrogen (TN). VWC, SBD, and TN were found to be the most significant factors affecting vegetation diversity under grazing. Partial canonical ordination revealed that soil moisture, soil bulk density, and SOC had the most significant correlation with vegetation diversity under both grazing prohibition and increased precipitation treatment conditions. These results suggested that alterations in precipitation can change the species composition and ecosystem function of plant communities, leading to a close relationship between species diversity and soil properties. Our results underline the importance of species diversity for soil characteristics and environmental factors in a typical Eurasian steppe ecosystem.