This study assessed the influence of rhizocompartment types (i.e., root, rhizosphere soil, root‐zone soil, and intershrub bulk soil) on the diversity of soil microbial communities under desert leguminous plant shrubs. Moreover, the influence and variations of soil physicochemical factors in interactions among leguminous plants, soil, and microbes were investigated. Both 16S rRNA high‐throughput genome sequencing and conventional soil physicochemical index determination were used to characterize both the bacterial diversity and soil physicochemical properties in the rhizocompartments of two
Hedysarum
species (
Hedysarum mongolicum
and
Hedysarum scoparium
) in the Mu Us Desert of China. All nutrient indices (except total phosphorus and available phosphorus) in rhizosphere soil were uniformly higher than those in both root‐zone soil and intershrub bulk soil (
p
< .05). The bacterial community diversity in the root, undershrub soil (i.e., rhizosphere and root zone), and intershrub bulk soil also showed significant differences (
p
< .05). The bacterial community in the root is mainly composed of Proteobacteria, Actinobacteria, Bacteroidetes, Tenericutes, and Chloroflexi, among which bacteria of the Proteobacteria genus are dominant. Root endophyte and rhizosphere soil microbiomes were mainly influenced by soil nutrients, while bacterial communities in root‐zone soil and intershrub bulk soil were mainly influenced by soil pH and NH
4
+
‐N. The rhizocompartment types of desert leguminous plants impose a significant influence on the diversity of soil microbial communities. According to these findings, nitrogen‐fixing rhizobia can co‐exist with nonsymbiotic endophytes in the roots of desert leguminous plants. Moreover, plants have a hierarchical filtering and enriching effect on beneficial microbes in soil via rhizocompartments. Soil physicochemical factors have a significant influence on both the structure and composition of microbial communities in various rhizocompartments, which is derived from the interactions among leguminous plants, soil, and microbes.