Large mammalian herbivores are well known to shape the structure and function of ecosystems world‐wide, and these effects can in turn cascade through systems to indirectly influence other animal species. A wealth of studies has explored the effects of large mammals on arthropods, but to date they have reported such widely varying results that generalizations have been elusive. Three factors are likely drivers of this variability: the widely varying life‐history characteristics of different arthropod groups, the highly variable landscapes that mammalian herbivores commonly inhabit and temporal variation in environmental conditions. Here, we use an 18‐year‐old exclosure experiment stratified across three distinct coastal prairie habitats in northern California to address the effects of a reintroduced mammalian herbivore, tule elk (Cervus canadensis nannodes) on the composition, richness and abundance of ground‐dwelling arthropods over two years with very different precipitation regimes. We found that elk shifted the composition of arthropod communities, increasing the abundance of ants, beetles, spiders and mites, decreasing the abundance of woodlice and bristletails in some but not all habitats types, and having no effect on the abundance of bugs, crickets and springtails. Elk also increased richness and changed the composition of ant genera and beetle morpho‐species. Interestingly, the effects of elk on arthropod composition, richness and abundance varied little between years, despite very different precipitation levels, biomass accumulation and thatch height. Elk reduced shrub cover, above‐ground herbaceous biomass and thatch height and increased soil compaction, and these changes predicted the abundance and richness of arthropods, although taxonomic groups varied in their responses, presumably due to differences in environmental requirements. Synthesis. Our research highlights the importance of using long‐term experiments to assess the cascading effects of large herbivores on the composition of grounddwelling arthropod communities and to identify the mechanisms that indirectly shape arthropod responses to herbivores among variable habitats and years in order to develop a greater understanding of the variable responses of arthropods to large mammalian herbivores.