Grassland degradation caused by increases in livestock grazing threatens a variety of ecosystem services. Understanding changes in plant community assembly during the process of grassland degradation in the presence of grazing is important to help restore degraded grasslands worldwide but has received little attention thus far. The grassland degradation process is typified by heterogeneous degradation, that is, gradual formation of degraded patches (hereafter “patchy degradation”). Here, we experimentally examined the effects of herbivore grazing and patchy degradation on plant community assembly using nine pairs of non‐degraded (intact) and patch‐degraded (fragmented) grasslands subject to grazing by different‐sized herbivores (i.e., NG, no grazing; SG, sheep grazing; CG, cattle grazing) over 4 years. Using a null‐model approach, we estimated the relative magnitude of deterministic processes of community assembly by comparing the observed and expected β‐diversity. We found that in the absence of herbivore grazing, deterministic processes played a greater role in community assembly, regardless of whether patchy degradation had occurred. However, the deterministic processes resulted in plant communities being more spatially similar in non‐degraded grasslands while being more dissimilar in patchy degraded grasslands. Compared with non‐degraded grasslands, species with strong competitive abilities (i.e., Leymus chinensis) were less dominant in patchy degraded grasslands, indicating relaxed competition and a reduced role of species interactions over plant communities. Instead, patchy degradation added the role of environmental variables over plant communities. SG consistently promoted more stochastic plant community assembly in both non‐degraded and patch‐degraded grasslands, while CG promoted more stochastic plant community assembly only in the non‐degraded state, having no effect in the patch‐degraded state. Our study offers important insights into changes in plant community assembly during ongoing patch‐degradation of grasslands, indicating the role of increased environmental filtering of soil and reduced species interactions in driving plant community dynamics with increasing grassland patchy degradation. We also uncovered an herbivore species‐specific effect on plant community assembly during the process of grassland degradation, which will better inform and improve future grassland restoration planning efforts.