Summary Afforestation of sandy arable soils in northern Europe is likely to lead to an increase in the soil's acidity and changes in the behaviour of the organic matter, and this might affect the ability of the soil to retain heavy metals. It is important to assess the impact of such a change in the land use on the solubility of the heavy metals and to assess the risk of leaching to surface‐ and groundwater and the possible entrapment of heavy metals in the tree canopy. The impact of afforestation was assessed by excavating soil profiles in adjacent 34‐year‐old Norway spruce stands and arable plots at four different sites. We found that after 34 years the pH had decreased and cations were depleted in the topsoil under forest. The aqua regia‐extractable heavy metals were determined, and the heavy metal binding within the soil was assessed using a modified version of the BCR (Community Bureau of Reference) sequential extraction procedure. Higher contents of heavy metal were found in the arable plots in the loamy sand soils. Cadmium was found only in the most mobile fractions. The content of Pb in the subsoil was strongly correlated with the clay content, but not in the topsoil, which suggested that Pb had been added to the topsoil. We found strong correlations between the clay content and the Cu, Ni and Zn in the residual fraction, leading us to conclude that much of the Cu, Ni and Zn is of geological origin. No significant differences in the heavy metal fractionation between forest and arable soil were found, presumably because 34 years of different land use is not long enough to produce such differences.