Plants take up a wide range of trace metals/metalloids (hereinafter referred to as trace metals) from the soil, some of which are essential but become toxic at high concentrations (e.g., Cu, Zn, Ni, Co), while others are non-essential and toxic even at relatively low concentrations (e.g., As, Cd, Cr, Pb, and Hg). Soil contamination of trace metals is an increasing problem worldwide due to intensifying human activities. Trace metal contamination can cause toxicity and growth inhibition in plants, as well as accumulation in the edible parts to levels that threatens food safety and human health. Understanding the mechanisms of trace metal toxicity and how plants respond to trace metal stress is important for improving plant growth and food safety in contaminated soils. The accumulation of excess trace metals in plants can cause oxidative stress, genotoxicity, programmed cell death, and disturbance in multiple physiological processes. Plants have evolved various strategies to detoxify trace metals through cell-wall binding, complexation, vacuolar sequestration, efflux, and translocation. Multiple signal transduction pathways and regulatory responses are involved in plants challenged with trace metal stresses. In this review, we discuss the recent progress in understanding the molecular mechanisms involved in trace metal toxicity, detoxification, and regulation, as well as strategies to enhance plant resistance to trace metal stresses and reduce toxic metal accumulation in food crops.