The levels of chlorophyll a and nutrient concentrations in the surface waters of the western subtropical Pacific Ocean are among the lowest globally. In addition, our knowledge of basin-scale diversity and biogeography of microbial communities in this vast extremely oligotrophic environment is still rather limited. Here, high-throughput sequencing was used to examine the biodiversity and biogeography of abundant and rare microbial assemblages throughout the water column from the surface to a depth of 3,000 m across a horizontal distance of 1,100 km in the western Pacific Ocean. Microbial alpha diversity in the 200-m layer was higher than at other depths, with Gammaproteobacteria, Alphaproteobacteria, and Clostridia as the dominant classes in all samples. Distinctly vertical distributions within the microbial communities were revealed, with no difference horizontally. Some microbes exhibited depth stratification. For example, the relative abundances of Cyanobacteria and Alphaproteobacteria decreased with depth, while Nitrososphaeria, Actinobacteria, and Gammaproteobacteria increased with depth in the aphotic layers. Furthermore, we found that environmental (selective process) and spatial (neutral process) factors had different effects on abundant and rare taxa. Geographical distance showed little effect on the dispersal of all and abundant taxa, while statistically significant distance–decay relationships were observed among the rare taxa. Temperature and chlorophyll a were strongly associated with all, abundant, and rare taxa in the photic layers, while total inorganic nitrogen was recognized as the crucial factor in the aphotic layers. Variance partitioning analysis indicated that environmental selection played a relatively important role in shaping all and abundant taxa, while the variation in rare taxa explained by environmental and spatial processes was relatively low, as more than 70% of the variation remained unexplained. This study provides novel knowledge related to microbial community diversity in the western subtropical Pacific Ocean, and the analyzes biogeographical patterns among abundant and rare taxa.