The use of large quantities of phosphorus-containing fertilizers has resulted in an increase in phosphorus content in the groundwater system, and phosphorus can be adsorbed on the surface of carbonate rocks, affecting their dissolution process and thus carbon sequestration and sink enhancement in carbonate rocks. Therefore, in this study, limestone was exposed to 2 mg/L and 100 mg/L phosphate solutions for 12 d through static batch adsorption experiments. The hydrochemical results showed that in 100 mg/L phosphate solution, a substitution reaction occurred to produce CaHPO4 precipitate, while the concentration of each ion in 2 mg/L phosphate solution was relatively stable and in dynamic equilibrium; combined with XRD and XPS analyses, the main mechanism of phosphate adsorption may be chemical precipitation, which is preferentially adsorbed to Ca sites on carbonate rocks, and the surface deposits are mainly CaHPO4 and a small amount of Mg2PO4(OH). The FTIR spectra were obtained in the range of 1040 cm−1–1103 cm−1 for observed phosphate vibrations, and the υ3 (asymmetric stretching) mode was more significant in the experimental group with a higher phosphate concentration. Raman spectra located near 149 cm−1 and 275 cm−1 involved Ca2+ or Mg2+ relative translations and vibrations, corroborating the FTIR spectroscopic results; a combination of XRD, XPS, FTIR, and Raman spectra confirmed that phosphate adsorption on limestone may be due to the interaction of electrostatic, chemical precipitation, and ligand exchange mechanisms. In addition, the SEM-EDS results showed that, with the combined effect of the water–rock chemical reaction and physical adsorption, metal–phosphorus phase precipitation was formed on the limestone surface, which promoted the dissolution of limestone and may have an unfavorable effect on the carbon sequestration and sinking of carbonate rocks.