The Cerrado biome is one of the global hotspots of biodiversity, and non-volant small mammals represent a significant portion of Cerrado species richness (45%) and endemism (86%). Nevertheless, we still lack a comprehensive picture of small mammal diversity patterns and drivers throughout the Cerrado. Here we surveyed small mammals across 45 sites to address species richness, abundance, and composition patterns and their drivers within and across sites, habitats, and localities at the world’s most diverse tropical savanna. As hypothesized, we found: (1) rich assemblages (12–21 species) characterized by few abundant and several intermediate-level and rare species; dominated by oryzomyine and akodontine cricetid rodents, and thylamyine and marmosine within marsupials, each tribe showing distinct habitat requirements; (2) strong habitat selectivity, with assemblages composed of forest dwellers, savanna specialists, and grassland inhabitants; and (3) similar species richness (α-diversity) but high species turnover (β-diversity) across sites, habitats, and localities, suggesting that horizontal stratification (within localities) and geographic location (across the Cerrado) are key drivers of small mammal diversity in tropical savannas. Thus, habitat heterogeneity and geographic location can be inferred as the main factors shaping species richness, abundance, and composition across the analyzed multiple spatial scales. Moreover, we found that geographical distance as well as the distance to neighbor biomes better explained species turnover, indicating landscape history and phylogenetic constraints as the major determinants of Cerrado small mammal diversity, as also evidenced for plants and other animal groups. These data highlight the need to preserve the mosaic of habitats across the different regions of the biome to conserve most of the Cerrado biodiversity.