The ecological niche gradient is an important determinant of microbial community structure. In this paper, we studied variation in rhizosphere bacterial diversity and community composition along an ecological niche gradient. We used the high-throughput sequencing of 16S rRNA genes to study changes in the rhizosphere soil microbial communities of six grass and four shrub species during the secondary succession of abandoned farmland on the Loess Plateau of China. A structural equation model (SEM) was employed to disentangle the relative contribution of ecological niche and soil properties to bacterial diversity and community composition. Proteobacteria, Acidobacteria, and Actinobacteria were the dominant phyla of rhizosphere bacteria in all samples. During the dynamics of the plant niche from low to high, bacterial community composition transitioned from Actinobacteria + Acidobacteria to Proteobacteria + Bacteroidetes higher abundance. Moreover, the bacterial diversity and species richness changed with an increasing niche gradient, showing a clear differentiation in the rhizosphere bacterial community of grassland and shrubland. Further, diversity and species richness decreased from the middle niche of B. ischaemum to the poles, indicating that the succession process had not yet reached the climax community stage. Community assembly analysis suggested that the stochastic process gradually strengthened along the increasing ecological niche gradient, especially the drift effect. Furthermore, SEM analysis showed that the ecological niche had significant negative effects on soil properties and bacterial richness, while the effects on bacterial diversity and the stochastic processes of community assembly were weakened and insignificant. Altogether, our findings suggest that the complex interaction of the ecological niche with bacterial diversity and composition was determined by soil properties. Further, bacterial diversity was not necessarily higher with increasing ecological niche gradients.