In the context of secondary forest succession, aboveground-belowground interactions are known to affect the dynamics and functional structure of plant communities. However, the links between soil microbial communities, soil abiotic properties, plant functional traits in the case of semi-arid and arid ecosystems, are unclear. In this study, we investigated the changes in soil microbial species diversity and community composition, and the corresponding effects of soil abiotic properties and plant functional traits, during a ≥150-year secondary forest succession on the Loess Plateau, which represents a typical semi-arid ecosystem in China. Plant community fragments were assigned to six successional stages: 1–4, 4–8, 8–15, 15–50, 50–100, and 100–150 years after abandonment. Bacterial and fungal communities were analyzed by high-throughput sequencing of the V4 hypervariable region of the 16S rRNA gene and the internal transcribed spacer (ITS2) region of the rRNA operon, respectively. A multivariate variation-partitioning approach was used to estimate the contributions of soil properties and plant traits to the observed microbial community composition. We found considerable differences in bacterial and fungal community compositions between the early (S1–S3) and later (S4–S6) successional stages. In total, 18 and 12 unique families were, respectively, obtained for bacteria and fungi, as indicators of microbial community succession across the six stages. Bacterial alpha diversity was positively correlated with plant species alpha diversity, while fungal diversity was negatively correlated with plant species diversity. Certain fungal and bacterial taxa appeared to be associated with the occurrence of dominant plant species at different successional stages. Soil properties (pH, total N, total C, NH
4
-N, NO
3
-N, and PO
4
-P concentrations) and plant traits explained 63.80% and 56.68% of total variance in bacterial and fungal community compositions, respectively. These results indicate that soil microbial communities are coupled with plant communities via the mediation of microbial species diversity and community composition over a long-term secondary forest succession in the semi-arid ecosystem. The bacterial and fungal communities show distinct patterns in response to plant community succession, according to both soil abiotic properties and plant functional traits.