The Dry-Hot Valley is a unique geographical region in southwestern China, where steep-slope cultivation and accelerating changes in land-use have resulted in land degradation and have aggravated soil erosion, with profound impacts on soil fertility. Soil microbes play a key role in soil fertility, but the impact of land-use changes on soil microbes in the Dry-Hot Valley is not well known. Here, we compared characteristics and drivers of soil microbial community composition and soil fertility in typical Dry-Hot Valley land uses of sugarcane land (SL), forest land (FL), barren land (BL) converted from former maize land (ML), and ML control. Our results showed that BL and SL had reduced soil organic carbon (SOC), total nitrogen (TN), and total potassium (TK) compared to ML and FL. This indicated that conversion of ML to SL and abandonment of ML had the potential to decrease soil fertility. We also found that fungal phyla Zoopagomycota and Blastocladiomycota were absent in SL and BL, respectively, indicating that land-use change from ML to SL decreased the diversity of the bacterial community. Redundancy analysis indicated that the relative abundance of bacterial phyla was positively correlated with TN, SOC, and available potassium (AK) content, and that fungal phyla were positively correlated with AK. Land-use indirectly affected the relative abundance of bacterial phyla through effects on soil moisture, clay, and AK contents, and that of fungal phyla through effects on clay and AK contents. In addition, land-use effects on bacteria were greater than those on fungi, indicating that bacterial communities were more sensitive to land-use changes. Management regimes that incorporate soil carbon conservation, potassium addition, and judicious irrigation are expected to benefit the stability of the plant–soil system in the Dry-Hot Valley.