AimMountains are geographic features that provide sharp elevational gradients which can accommodate a diversity of terrestrial flora and fauna. In the face of climate change, many of these species are being pushed higher to escape ever‐increasing temperatures. Despite this, we have little understanding of how species distribute themselves across mountains. African mountains in particular are less studied than mountains elsewhere and the small mammal groups that inhabit them are also poorly studied when compared to other groups. In this study, we examined the responses of three functionally distinct small mammal taxonomic groups to changes in elevation across Africa.LocationAfrica.MethodsWe calculated four functional diversity metrics for 166, 97 and 153 communities of rodents, bats and shrews comprising 225, 183 and 109 species, respectively. We employed RLQ and fourth‐corner analyses to identify any associations between the functional traits of the small mammals and elevation and other climatic variables.ResultsWe show that the species richness of the three taxa responded differently to increases in elevation, with only rodents showing a mid‐elevational hump. The composition of rodent and bat communities differed significantly in upland versus lowland sites, but bat communities showed far more overlap across the elevational gradient. Functional diversity metrics responded differently for each taxon highlighting the importance of using different taxonomic groups when studying elevational patterns of functional diversity.Main ConclusionsOur study shows that functional divergence increases with elevation in all three taxa, indicating a pervasive and broadly applicable strong environmental filtering at higher elevations. Our findings suggest that species at higher elevations may be at higher risk due to specialization. These species are losing habitat due to ongoing climate change that will force them up the elevational gradient. At the same time, the human population in Africa is predicted to triple in size by the year 2100, which will also likely contribute to further habitat loss and fragmentation. As small mammals often play essential roles in ecosystems, from seed dispersal to soil aeration and predation, understanding the susceptibility of the specialization of species at higher elevations is crucial for the effective conservation and management of mountain ecosystems, especially in light of climatic change and human expansion.