There has been increasing interest in using biochar for nutrient removal from water, and its application for anionic nutrient removal such as in phosphate (PO43−) necessitates surface modifications of raw biochar. This study produced avocado seed biochar (AB), impregnated Fe- or Mg-(hydr)oxide onto biochar (post-pyrolysis), and tested their performance for aqueous phosphate removal. The Fe- or Mg-loaded biochar was prepared in either high (1:8 of biochar to metal salt in terms of mass ratio) or low (1:2) loading rates via the co-precipitation method. A total of 5 biochar materials (unmodified AB, AB + High Fe, AB + Low Fe, AB + High Mg, and AB + Low Mg) were characterized according to their selected physicochemical properties, and their phosphate adsorption performance was tested through pH effect and adsorption isotherm experiments. Fe-loaded AB contained Fe3O4, while Mg-loaded AB contained Mg(OH)2. The metal (hydr)oxide inclusion was higher in Fe-loaded AB. Mg-loaded AB showed a unique free O–H functional group, while Fe-loaded AB showed an increase in its specific surface area more than 10-times compared to unmodified AB (1.8 m2 g−1). The effect of the initial pH on phosphate adsorption was not consistent between Fe-(anion adsorption envelope) vs. Mg-loaded AB. The phosphate adsorption capacity was higher with Fe-loaded AB in low concentration ranges (≤50 mg L−1), while Mg-loaded AB outperformed Fe-loaded AB in high concentration ranges (75–500 mg L−1). The phosphate adsorption isotherm by Fe-loaded AB fit well with the Langmuir model (R2 = 0.91–0.96), indicating the adsorptive surfaces were relatively homogeneous. Mg-loaded biochar, however, fit much better with Freundlich model (R2 = 0.94–0.96), indicating the presence of heterogenous adsorptive surfaces. No substantial benefit of high loading rates in metal impregnation was found for phosphate adsorption. The enhanced phosphate removal by Mg-loaded biochar in high concentration ranges highlights the important role of the chemical precipitation of phosphate associated with dissolved Mg2+.