Understanding microbial communities associated with bulk and rhizosphere soils will benefit the maintenance of forest health and productivity and the sustainable development of forest ecosystems. Based on MiSeq sequencing, we explored the differences between the bulk soil and the rhizosphere soil on bacterial and fungal communities of pecan plantation. Results suggested that rhizosphere-associated fungal rather than bacterial community structures differed from bulk soil, and rhizosphere soil had lower fungal diversity than bulk soil. Actinobacteria and Cantharellales were the bacterial and fungal biomarkers of the rhizosphere soil of pecan plantation, respectively. In addition, Pleosporales, which are mainly involved in saprophylaxis and plant pathogenic processes, was identified as one of the most important fungal biomarkers for the bulk soil, and the FunGuild predicted a higher relative abundance of pathogenic fungi in bulk soil compared to rhizosphere soil. The pH, ammonium nitrogen (NH4+-N), nitrate nitrogen (NO3--N), and total carbon (TC) contents drove microbial community structure and composition. The bacterial network was simpler in the rhizosphere soil than in the bulk soil. However, fungi showed the opposite network pattern. Keystone species in bacterial and fungal networks were mostly involved in nutrient cycling and the C cycling, and were found to be enriched in the rhizosphere soil. Overall, in terms of bacterial and fungal communities, the rhizosphere soil behaves more healthily than the bulk soil and has a higher potential for nutrient cycling.