Soil microorganisms play vital roles in regulating multiple ecosystem functions. Recent studies have revealed that the rare microbial taxa (with extremely low relative abundances, which are still largely ignored) are also crucial in maintaining the health and biodiversity of the soil and may respond differently to environmental pressure. However, little is known about the soil community structures of abundant and rare taxa and their assembly processes in different soil layers on the Qinghai‐Tibet Plateau (QTP). The present study investigated the community structure and assembly processes of soil abundant and rare microbial taxa on the northeastern edge of the QTP. Soil microbial abundance was defined by abundant taxa, whereas rare taxa contributed to soil microbial diversity. The results of null model show that the stochastic process ruled the assembly processes of all sub‐communities. Dispersal limitation contributed more to the assembly of abundant microbial taxa in the different soil layers. In contrast, drift played a more critical role in the assembly processes of the rare microbial taxa. In addition, in contrast to previous studies, the abundant taxa played more important roles in co‐occurrence networks, most likely because of the heterogeneity of the soil, the sparsity of amplicon sequencing, the sampling strategy, and the limited samples in the present study. The results of this study improve our understanding of soil microbiome assemblies on the QTP and highlight the role of abundant taxa in sustaining the stability of microbial co‐occurrence networks in different soil layers.