The chemical reaction between calcium ions (Ca2+) and phosphate in the soil is the main way to maintain the availability of soil phosphorus. Thus, we believe stimulating coal gangue with Ca2+ solution would be an effective way to improve its adsorption and desorption capacity toward phosphate. In order to explore the effects of different pH of Ca2+ solution on the modified effect of coal gangue, we conducted mechanical grinding (<1 mm), high temperature calcination (800 °C), and the stimulation of Ca2+ solution with different pH (2, 7, 13), to prepare acidic calcium-modified coal gangue (Ac-CG) (Ac-CG, acidic calcium-modified coal gangue; Ne-CG, neutral calcium-modified coal gangue; Al-CG, alkali calcium-modified coal gangue; RCG, raw coal gangue), neutral calcium-modified coal gangue (Ne-CG), and alkali calcium-modified coal gangue (Al-CG); raw coal gangue (RCG) was regarded as the control. The results indicated that Al-CG had better phosphate adsorption (3.599 mg g−1); this favorable adsorption performance of Al-CG was related to the formation of hydrated calcium silicate gel and ettringite, which provided more Ca2+, Al3+, and hydroxyl groups, and a larger specific surface area (9.497 m2 g−1). Moreover, Al-CG not only held more phosphate but also maintained its availability longer for plants. It is suggested that stimulating coal gangue with Ca2+ solution under alkaline conditions is a perfect way to enhance its adsorption and desorption capacity toward phosphate; the Al-CG we prepared could be used as filling material and soil conditioner in the reclamation area.