In this work, ternary and quaternary layered double oxides (PTB-LDO3 and PTB-LDO4) based on phosphate tailings were synthesized by the coprecipitation method. The as-prepared samples were characterized and applied to remove fluorine ions from a phosphogypsum leaching solution. The results indicated that both the precursor of PTB-LDO3 and PTB-LDO4 showed a layered structure with characteristic diffraction peaks of hydrotalcite. Compared with PTB-LDO4, PTB-LDO3 exhibited better adsorption performance at pH 5–6 and a dosage of 0.04 mg L−1. The adsorption kinetics results revealed that the adsorption of fluorine by PTB-LDO3 and PTB-LDO4 reached the adsorption equilibrium in about 3 h, and followed the pseudo-second-order model. The adsorption data could be fitted better with the Langmuir isotherm with the maximum adsorption amounts of 26.03 mg g−1 and 15.66 mg g−1 for PTB-LDO3 and PTB-LDO4, respectively. The adsorption of fluorine by PTB-LDO3 and PTB-LDO4 were both spontaneous and exothermic, and exhibited excellent reusability and stability. This study provides a possibility for the combined treatment of phosphorus chemical solid waste (phosphorus tailings) and phosphorus chemical wastewater (phosphogypsum leaching liquid).