Soil erosion on slope farmland causes the degradation of soil quality and eutrophication of water bodies due to the loss of phosphorus. In order to explore the influence of soil mechanical composition and mineral composition on phosphorus fixation, we selected calcareous soil on slope farmland in the Three Gorges Reservoir area as the research object and separated the samples on the basis of particle size. Next, we determined the content of different forms of phosphorus for each particle size, and then characterized and analyzed the mineral composition. The adsorption performance of each particle size was also studied. The results show that the calcareous soil on slope farmland has a high proportion of coarse fractions, and the carbonate minerals enriched in coarse fractions will fix a large amount of phosphorus and degrade soil quality. As slope farmland is prone to soil erosion, when the soil undergoes selective migration, the loss of fine fractions will increase the proportion of coarse fractions and aggravate soil quality degradation. Meanwhile, because of the large amount of phosphorus adsorbed by fine fractions, the available phosphorus is easily lost with fine fractions, leading to eutrophication of water bodies.