The decontamination of water containing toxic metals is a challenging problem, and in the last years many efforts have been undertaken to discover efficient, cost-effective, robust, and handy technology for the decontamination of downstream water without endangering human health. According to the World Health Organization (WHO), 180 million people in the world have been exposed to toxic levels of arsenic from potable water. To date, a variety of techniques has been developed to maintain the arsenic concentration in potable water below the limit recommended by WHO (10 μg/L). Recently, a series of technological advancements in water remediation has been obtained from the rapid development of nanotechnology-based strategies that provide a remarkable control over nanoparticle design, allowing the tailoring of their properties toward specific applications. Among the plethora of nanomaterials and nanostructures proposed in the remediation field, graphene-based materials (G), due to their unique physico-chemical properties, surface area, size, shape, ionic mobility, and mechanical flexibility, are proposed for the development of reliable tools for water decontamination treatments. Moreover, an emerging class of 3D carbon materials characterized by the intrinsic properties of G together with new interesting physicochemical properties, such as high porosity, low density, unique electrochemical performance, has been recently proposed for water decontamination. The main design criteria used to develop remediation nanotechnology-based strategies have been reviewed, and special attention has been reserved for the advances of magnetic G and for nanostructures employed in the fabrication of membrane filtration.