In this study, magnetic Fe3O4 particles and Fe3O4-Ag0 nanocomposites were prepared by a facile and green method, fully characterized and used for the removal of Hg2+ from water. Characterizations showed that the Fe3O4 particles are quasi-spherical with an average diameter of 217 nm and metallic silver nanoparticles formed on the surface with a size of 23–41 nm. The initial Hg2+ removal rate was very fast followed by a slow increase and the maximum solid phase loading was 71.3 mg/g for the Fe3O4-Ag0 and 28 mg/g for the bare Fe3O4. The removal mechanism is complex, involving Hg2+ adsorption and reduction, Fe2+ and Ag0 oxidation accompanied with reactions of Cl− with Hg+ and Ag+. The facile and green synthesis process, the fast kinetics and high removal capacity and the possibility of magnetic separation make Fe3O4-Ag0 nanocomposites attractive materials for the removal of Hg2+ from water.