Increasing industrialization and urbanization result in emission of pollutants in the environment including toxic heavy metals, as cadmium and lead. Among the different heavy metals contaminating the environment, cadmium raises great concern, as it is ecotoxic and as such can heavily impact ecosystems. The cell wall is the first structure of plant cells to come in contact with heavy metals. Its composition, characterized by proteins, polysaccharides and in some instances lignin and other phenolic compounds, confers the ability to bind non-covalently and/or covalently heavy metals via functional groups. A strong body of evidence in the literature has shown the role of the cell wall in heavy metal response: it sequesters heavy metals, but at the same time its synthesis and composition can be severely affected. The present review analyzes the dual property of plant cell walls, i.e., barrier and target of heavy metals, by taking Cd toxicity as example. Following a summary of the known physiological and biochemical responses of plants to Cd, the review compares the wall-related mechanisms in early- and later-diverging land plants, by considering the diversity in cell wall composition. By doing so, common as well as unique response mechanisms to metal/cadmium toxicity are identified among plant phyla and discussed. After discussing the role of hyperaccumulators’ cell walls as a particular case, the review concludes by considering important aspects for plant engineering.