Metal hyperaccumulation in plants is an ecological trait whose biological significance remains debated, in particular because the selective pressures that govern its evolutionary dynamics are complex. One of the possible causes of quantitative variation in hyperaccumulation may be local adaptation to metalliferous soils. Here, we explored the population genetic structure of Arabidopsis halleri at fourteen metalliferous and nonmetalliferous sampling sites in southern Poland. The results were integrated with a quantitative assessment of variation in zinc hyperaccumulation to trace local adaptation. We identified a clear hierarchical structure with two distinct genetic groups at the upper level of clustering. Interestingly, these groups corresponded to different geographic subregions, rather than to ecological types (i.e., metallicolous vs. nonmetallicolous). Also, approximate Bayesian computation analyses suggested that the current distribution of A. halleri in southern Poland could be relictual as a result of habitat fragmentation caused by climatic shifts during the Holocene, rather than due to recent colonization of industrially polluted sites. In addition, we find evidence that some nonmetallicolous lowland populations may have actually derived from metallicolous populations. Meanwhile, the distribution of quantitative variation in zinc hyperaccumulation did separate metallicolous and nonmetallicolous accessions, indicating more recent adaptive evolution and diversifying selection between metalliferous and nonmetalliferous habitats. This suggests that zinc hyperaccumulation evolves both ways-towards higher levels at nonmetalliferous sites and lower levels at metalliferous sites. Our results open a new perspective on possible evolutionary relationships between A. halleri edaphic types that may inspire future genetic studies of quantitative variation in metal hyperaccumulation.