Studies on elevational gradients in biodiversity have accumulated in recent decades. However, few studies have compared the elevational patterns of diversity between the different slopes of a single mountain. We investigated the elevational distribution of rodent diversity (alpha and beta diversity) and its underlying mechanisms along the southern and northern slopes of Mt. Taibai, the highest mountain in the Qinling Mountains, China. The species richness of rodents on the two slopes showed distinct distribution patterns, with a monotonically decreasing pattern found along the southern slope and a hump-shaped elevational pattern evident along the northern slope. Multi-model inference suggested that temperature was an important explanatory factor for the richness pattern along the southern slope, and the mid-domain effect (MDE) was important in explaining the richness pattern along the northern slope. The two slopes also greatly differed in the elevational patterns of species turnover, with the southern slope demonstrating a U-shaped curve and the northern slope possessing a roughly hump-shaped pattern. Our results suggest that even within the same mountain, organisms inhabiting different slopes may possess distinct diversity patterns, and the underlying mechanisms may also differ. The potential role of the factors associated with slope aspect in shaping diversity, therefore, cannot be ignored.