PPy-Fe3O4/Kaolin was prepared with polypyrrole functionalized magnetic Kaolin by a simple, green, and low cost method to improve the agglomeration and low adsorption capacity of Kaolin. PPy-Fe3O4/Kaolin was employed to remove Hg2+ and the results were characterized by various methods. Relevant factors, including solution pH, dosage of adsorbent, concentration (C0), and temperature (T), were optimized by Response Surface Methodology (RSM) and Central Composite Designs (CCD). The optimal results show that the importance for adsorption factors is pH > T > C0 > dosage, and the optimal adsorption conditions of PPy-Fe3O4/Kaolin are pH = 7.2, T = 315 K, C0 = 50 mg/L, dosage of 0.05 g/L, and the capacity is 317.1 mg/g. The adsorption process conforms to the pseudo-second-order and Langmuir models. Dubinin–Radushkevich model shows that adsorption process is spontaneous and endothermic. Moreover, the adsorption of mercury by PPy-Fe3O4/Kaolin was achieved mainly through electrostatic attraction, pore diffusion, and chelation between amino functional groups and Hg2+. PPy-Fe3O4/Kaolin has excellent reproducibility, dispersity, and chemical stability, and it is easy to be separated from solution through an external magnetic field. The experiments show that PPy-Fe3O4/Kaolin is an efficient and economical adsorbent towards mercury.