The mammalian gut microbiota is considered pivotal to host fitness, yet the determinants of community composition remain poorly understood. Laboratory studies show that environmental factors, particularly diet, are important, while comparative work emphasises host genetics. Here, we compare the influence of host genetics and the environment on the microbiota of sympatric small mammal species (mice, voles, shrews) across multiple habitats. While sharing a habitat caused some microbiota convergence, the influence of species identity dominated. In all three host genera examined, an individual's microbiota was more similar to conspecifics living elsewhere than to heterospecifics at the same site. Our results suggest this species‐specificity arises in part through host‐microbe codiversification. Stomach contents analysis suggested that diet also shapes the microbiota, but where diet is itself influenced by species identity. In this way, we can reconcile the importance of both diet and genetics, while showing that species identity is the strongest predictor of microbiota composition.