Water sources in arid and semiarid ecosystems support humans, wildlife, and domestic animals, forming nodes of activity that sculpt surrounding plant communities and impact critical grazing and soil systems. However, global aridification and changing surface water supply threaten to disrupt these water resources, with strong implications for conservation and management of these ecosystems. To understand how effects of herbivore aggregation at water impact plant communities across contexts, we measured herbivore activity, plant height, cover (trees, grasses, forbs, and bare ground), diversity, and composition at 17 paired water sources and matrix sites across a range of abiotic factors in a semiarid savanna in Kenya. The effects of proximity to surface water and herbivore aggregation on plant communities varied substantially depending on soil and rainfall. In arid areas with nutrient‐poor sandy soils, forb and tree cover were 50% lower at water sources compared to neighboring matrix sites, bare ground was 20% higher, species richness was 15% lower, and a single globally important grazing grass (Cynodon dactylon) dominated 60% of transects. However, in mesic areas with nutrient‐rich finely textured soils, species richness was 25% higher, despite a 40% increase in bare ground, concurrent with the decline of a dominant tall grass (Themeda triandra) and increase in C. dactylon and other grass species near water sources. Recent rainfall was important for grasses; cover was higher relative to matrix sites only during wet periods, a potential indication of compensatory grazing. These findings suggest that effects of herbivore aggregation on vegetation diversity and composition will vary in magnitude, and in some cases direction, depending on other factors at the site. Where moisture and nutrient resources are high and promote the dominance of few plant species, herbivore aggregations may maintain diversity by promoting grazing lawns and increasing nondominant species cover. However, in arid conditions and sites with low nutrient availability, diversity can be substantially reduced by these aggregations. Our results highlight the importance of considering abiotic conditions when managing for effects of herbivore aggregations near water. This will be particularly important for future managers in light of growing global aridification and surface water changes.