Ultramafic soils are characterized by severe edaphic conditions induced by a low content of essential nutrients, an adverse Ca/Mg ratio, a low water-holding capacity, and high contents of geogenic potentially toxic elements (PTEs), in particular Cr, Ni, and Co. These metals commonly exceed the content limits set by environmental agencies and governments, representing serious environmental risks for ecosystems and human health. In alpine environments, ultramafic soils are characterized by modest thickness and poor horizon differentiation. Several studies on ultramafic soils have shown that their properties may be directly related to the characteristics of the parent rocks, but most of these studies deal with soil chemistry, metal availability, isotopic composition, and pedological characterization. The aim of this research is to investigate how much the geotectonic characteristics of ultramafic bedrocks, such as the degree of serpentinization, metamorphic imprint, and deformation, may affect the mineralogical and chemical variations of ultramafic soils, including the occurrence and potential mobility of the PTEs. Using a multiscale and multi-analytical approach, we fully characterize the properties and mineralogical composition of soil profiles with different ultramafic parent rocks, i.e., partially serpentinized peridotite, massive serpentinites, and foliated serpentinites, sampled within the Voltri Massif High Pressure–Low Temperature (HP–LT) metaophiolite (Western Alps, Italy). Our results, related to soils located at comparable latitude, altitude, landscape position, and pedological environment, outline that the degree of serpentinization, the metamorphic imprint, and the deformation history of the ultramafic parent rocks are key factors influencing soil evolution, mineralogy, and chemistry, as well as PTEs distribution and mobility. Moreover, this study shows that the high content of Cr, Ni, and Co in the studied ultramafic soils has to be considered of geogenic origin and highlights the need for new approaches and methods to obtain indications on the potential contamination of natural or anthropogenic soils.