Three different herbivore grazing assemblages, namely, yak grazing (YG), Tibetan sheep grazing (SG) and yak and Tibetan sheep co-grazing (MG), are practiced in alpine meadows on the Qinghai-Tibetan Plateau (QTP), but the effects of the different herbivore assemblages on soil microbes are relatively unknown. The microbial community plays an important role in the functional stability of alpine grassland ecosystems. Therefore, it is important to understand how the microbial community structure of grassland ecosystems changes under different herbivore grazing assemblages to ensure their sustainable development. To fill this gap, a field study was carried out to investigate the effects of YG, SG, and MG on plant communities, soil physico-chemical properties and microbial communities under moderate grazing intensity in alpine meadows. Grazing increased the β-diversity of the bacteria community and decreased the β-diversity of the fungal community. The herbivore assemblage affected the microbial community diversity, but not the plant community diversity. Total phosphorus, soil bulk density, root biomass, and plant α-diversity were correlated with both the bacterial and fungal community composition, available phosphorus and soil moisture were correlated only with the bacterial community composition, while available potassium and above-ground net primary production (ANPP) were correlated only with the fungal community composition. Soil available nitrogen, soil available phosphorus and soil bulk density were highest in SG, while ANPP was highest in MG. It was concluded that MG can improve ANPP and stabilize the soil microbial community, suggesting that MG is an effective method for sustainable use and conservation of alpine meadows on the QTP.