Understanding the mechanisms influencing patterns and processes of biological diversity is critical to protecting biodiversity, particularly in species-rich ecosystems such as mountains. Even so, there is limited knowledge of biodiversity patterns and processes in the mountains of eastern China, especially about small mammals. In this study, we examined the taxonomic, functional, and phylogenetic diversity of small mammal distribution and community structure along the elevational gradient of Qingliang Mountain, eastern China. We then evaluated how they are influenced by space (area and mid-domain effect (MDE)), environment (temperature, precipitation, and normalized difference vegetation index (NDVI)), and human disturbance. The results showed hump-shaped patterns of taxonomic and phylogenetic diversity along elevation gradients, peaking at 1000 m, unlike functional diversity, which peaked at lower elevations (600 m). The mean pairwise distance and mean nearest taxon distance of functional and phylogenetic variance (MFD and MPD, respectively) were also incongruent. The MFD and MPD showed hump-shaped patterns along elevations; however, unlike MFD, which peaked at lower elevations (600 m), MPD peaked at higher elevations (1200 m). The mean nearest functional taxon distance (MNFD) decreased, while the mean nearest phylogenetic taxon distance (MNTD) increased along the elevation gradient. The higher elevations were functionally more clustered, while the lower elevations were phylogenetically more clustered, suggesting that environmental filtering for traits was stronger at higher elevations. In comparison, phylogenetic conservatism of ecological niches had a stronger influence at lower elevations. The diversity and community structure indices were inconsistently explained, with human disturbance and MDE accounting for the biggest proportions of the model-explained variances. Overall, the results confirm that environmental filtering and human disturbance significantly influence small mammals’ diversity and community structure. These findings also emphasize the need for increased conservation efforts in the middle and lower elevation regions of Qingliang Mountain.