Global warming is likely to affect the underground microbial communities in various ecosystems, but the response of soil microbial communities along a vertical depth profile to global warming has been elusive. Herein, we leveraged a warming field experiment in Qinghai-Tibet Plateau grassland and investigated the community structure of prokaryotes and fungi from the upper (0–15 cm) and lower (15–30 cm) strata under ambient and elevated temperature treatments. Three-years continual warming only significantly shifted the prokaryotic community within the upper strata and there was no significant effect observed for the fungal community. Additionally, under ambient temperature, there were significant differences between the two strata in both the prokaryotic and fungal communities, but under warming, this effect was alleviated. Next, the prokaryotic and fungal community assembly processes were measured by a phylogenetic-bin-based null approach (iCAMP). Though deterministic and stochastic processes dominated the assembly of prokaryotic and fungal communities, respectively, the deterministic processes were strengthened under warming for both communities. Specifically, the increased portion of homogeneous selection, contributing to a homogenous state, led to a smaller difference between prokaryotic communities of the two soil strata under warming. The smaller difference in the stochastic process component, i.e., dispersal limitation, contributed to the similar fungal community structures between the two strata under warming. This study deepens our understanding of warming effects on grassland microbial communities and gives greater insights on the underlying mechanisms for microbial assembly between upper and lower soil strata under warming scenarios.