Rhizosphere soil microbial communities substantially impact plant growth by regulating the nutrient cycle. However, dynamic changes in soil microbiota under different tree ages have received little attention. In this study, changes in soil physicochemical properties, as well as bacterial diversity and community structures (by high-throughput Illumina MiSeq sequencing), were explored in pomelo trees of different ages (i.e., 10, 20, and 30 years) under red and paddy soils cultivated by farmers with high fertilizer input. Moreover, soil factors that shape the bacterial community, such as soil pH, AP (available phosphorous), AK (available potassium), and AN (available nitrogen), were also investigated. Results showed that pH significantly decreased, while AP, AK, and AN increased with increasing tree age under red soil. For paddy soil, pH was not changed, while AP was significantly lower under 10-year-old pomelo trees, and AK and AN contents were minimum under 30-year-old pomelo trees. Both soil types were dominated by Proteobacteria, Acidobacteria, and Actinobacteria and showed contrasting patterns of relative abundance under different tree age groups. Bacterial richness and diversity decreased with increasing tree age in both soil types. Overall, bacterial community composition was different under different tree ages. RDA analysis showed that soil pH, AP, and AN in red soil, and pH and AP in paddy soil showed the most significant effects in changing the bacterial community structure. A random forest model showed Sinomonas and Streptacidiphilus in red soil, while Actinoallomurus and Microbacterium in paddy soil were the most important genera explaining the differences among different age groups. The ternary plot further revealed that genera enrichment for Age_30 was higher than that for Age_10 and Age_20 in red soil, whereas specific genera enrichment decreased with increasing tree age under paddy soil. Co-occurrence network revealed that bacterial species formed a complex network structure with increasing tree age, indicating a more stable microbial association under 20 and 30 years than 10-year-old pomelo trees. Hence, contrasting patterns of changes in soil physicochemical properties and soil microbial communities were recorded under different tree ages, and tree ages significantly affected the bacterial community structure and richness. These findings provide valuable information regarding the importance of microbes for the sustainable management of pomelo orchards by optimizing fertilizer input for different ages of trees.