From ancient times until recently, mining and metallurgical activities in the greater Lavrion area, Greece, have resulted in the production of large amounts of metal wastes that enrich the surrounding soils with toxic metal(loid)s. Since the area is inhabited, soil dust inhalation, consumption of vegetables grown on the contaminated soils, and of dairy products from small ruminants grazing in the area, may impose a real threat for the health of the local residents. After more than 3,000 years of mining history, this study is the first report on the chemical partitioning and behavior of Zn, Cu, Ni, Cr, Cd, Pb, and As in the topsoils (0–15 cm) of Lavrion. Community Bureau of Reference (BCR) sequential extraction was applied to 29 topsoil samples and 4 chemical fractions of the studied elements were defined (a) exchangeable/acid soluble (F1), (b) reducible (F2), (c) oxidisable (F3), and (d) residual (RF). Total concentrations of all elements were above the geochemical background. Zinc, Pb, Cd, and As median total concentration values were 4, 9, 1.4, and 17 times higher than the respective intervention thresholds, indicating severe soil pollution. Most of Cd (78.2%), Pb (69.7%), and Zn (53.3%) were found in F1 + F2 sum, suggesting high availability under modified conditions that periodically can occur in soils. Copper, Ni, As, and Cr were mainly associated with the residual fraction. Principal component analysis results showed that F2 fraction of all studied elements was associated to Fe oxides, confirming the crucial role of Fe oxides in metal(loid)s sequestration in the soil matrix.