Organic phosphorus (Po) may play a vital role in phosphorus availability via its mineralization by alkaline phosphatase (ALP), being encoded by phoD gene, in calcareous soil. Understanding the effects of long-term fertilization on the community of phoD harboring bacteria and the related alteration of the P availability owing to the changes in ALP secretion may offer a chance to elucidate the Po contribution to soil available P. Based on a long-term experiment, we analyzed the phoD gene harboring microbial diversity, abundance and composition, ALP and Po forms, and their relationship. The treatments involved were control without any fertilizers (CK), synthetic nitrogen and potassium (NK), synthetic nitrogen, phosphorus and potassium (NPK), NPK and crop stalk return (SNPK), and NPK plus organic manure (MNPK). Fertilization increased the abundance and diversity of phoD gene harboring microbial over CK. Those receiving NPK and NPK treatments integrated with organic supplements significantly improved the relative abundance of Proteobacteria but decreased Gemmatimonadetes at the phylum level, while all fertilized treatments appreciably increased the relative abundance of Lysobacter but decreased that of Gemmatirosa and Afipia, at the genus level. SNPK and MNPK treatments noticeably increased the relative abundance of Methylobacter but reduced Pseudomonas and Streptomyces relative to those receiving synthetic fertilizer treatments. Long-term fertilization markedly raised ALP activity, which was significantly and positively correlated with the relative abundance of the phylum Proteobacteria as represented by the genera Methylobacterium and Lysobacter. ALP was closely associated with moderately labile Po, followed by enzyme P, recalcitrant Po, and labile Po. The changes in phoD bacteria and ALP were mainly driven by soil organic carbon, Olsen P and pH. We concluded that the long-term fertilization, especially the addition of organic supplements, profoundly modified the soil properties and subsequently changed the diversity and relative abundance of phoD gene harboring bacteria, which promoted the activity of ALP, and thus the mineralization of various forms of Po (mainly moderately labile Po) to enhance the P availability.