Pb(II) contamination in aquatic environments has adverse effects on humans even at a low concentration, so the efficient removal of Pb at a low cost is vital for achieving an environmentally friendly, sustainable, and healthy society. A variety of CaCO3-based functional adsorbents have been synthesized to remove Pb, but the adsorption capacity is still unsatisfactory. Herein, calcite CaCO3 microcubes/parallelepipeds are synthesized via simple precipitation and a hydrothermal approach and found to outperform previously reported nano-adsorbents considerably. The CaCO3 achieves a high removal efficiency for Pb(II) (>99%) at a very low dosage (0.04–0.1 g/L) and an initial Pb(II) concentration of 100 mg/L. The CaCO3 presents an excellent adsorption capacity of 4018 mg/g for Pb(II) removal and depicts good stability over a wide range of pH 6–11. The maximum adsorption kinetics are fitted well by the pseudo-second-order kinetic model, whereas the Freundlich isotherm delineates the adsorption data at equilibrium well, indicating a multilayer adsorption process. The ex situ study confirms that the Pb(II) adsorption mechanism by CaCO3 can be attributed to the rapid metal-ion-exchange reaction between Pb(II) and Ca2+. Furthermore, a red shift in the Fourier Transform Infrared (FTIR) spectroscopy peak from 1386 cm−1 to 1374 cm−1 of CaCO3 after Pb removal indicates the adsorption of Pb onto the surface. This adsorbent provides an opportunity to treat wastewater and can be extended to remove other toxic heavy metals.