Phosphonates are widely used as antiscalants for softening processes in drinking water treatment. To prevent eutrophication and accumulation in the sediment, it is desirable to remove them from the membrane concentrate before they are discharged into receiving water bodies. This study describes batch experiments with synthetic solutions and real membrane concentrate, both in the presence of and absence of granular ferric hydroxide (GFH), to better understand the influence of ions on phosphonate and phosphate adsorption. To this end, experiments were conducted with six different phosphonates, using different molar Ca:phosphonate ratios. The calcium already contained in the GFH plays an essential role in the elimination process, as it can be re-dissolved, and, therefore, increase the molar Ca:phosphonate ratio. (Hydrogen-)carbonate ions had a competitive effect on the adsorption of phosphonates and phosphate, whereas the influence of sulfate and nitrate ions was negligible. Up to pH 8, the presence of CaII had a positive effect on adsorption, probably due to the formation of ternary complexes. At pH > 8, increased removal was observed, with either direct precipitation of Ca:phosphonate complexes or the presence of inorganic precipitates of calcium, magnesium, and phosphate serving as adsorbents for the phosphorus compounds. In addition, the presence of (hydrogen-)carbonate ions resulted in precipitation of CaCO3 and/or dolomite, which also acted as adsorbents for the phosphorus compounds.