Sediment, suspended matter (SM), and water of a large river (Neckar; River1) and a small river (Körsch; River2) were analyzed for the phosphonates 2-phosphonobutane-1,2,4-tricarboxylic acid (PBTC), 1-hydroxyethylidene (1,1-diphosphonic acid) (HEDP), aminotris (methylphosphonic acid) (ATMP), ethylenediaminetetra (methylene phosphonic acid) (EDTMP), and diethylenetriaminepenta (methylene phosphonic acid) (DTPMP). Ten samplings were performed at intervals of one to two months during one year, each covering the relevant matrices before and behind the discharge point of a wastewater treatment plant (WWTP). In River1, the total concentration of dissolved phosphonate did not change significantly (2.4–5.8 µg/L before vs. 2.5–6.6 µg/L behind WWTP; p = 0.9360). In River2, it increased significantly from <0.1–1.6 µg/L to 19–39 µg/L (p < 0.0001). Based on the median, the total phosphonate load in River1 sediment increased 1.9-fold (6.7–29.4 mg/kg before vs. 17.8–53.5 mg/kg behind WWTP; p = 0.0033) and in River2 by a factor of eight (1.8–5.0 mg/kg before vs. 18.1–51.4 mg/kg behind WWTP; p < 0.0001). This indicates that phosphonates discharged by WWTPs adsorb onto solid particles and accumulate in the sediment. In the case of River2, the SM load could reach values of 1000–1710 mg/kg behind the WWTP, presumably due to the introduction of insufficiently retained activated sludge particles of >2000 mg/kg phosphonate loads. In general, the nitrogen-free phosphonates PBTC and HEDP were most predominant in both dissolved and adsorbed form, of which HEDP had the highest adsorption affinity.