A wide genetic diversity has been reported for wild olives, which could be particularly interesting for the introgression of some agronomic traits and resistance to biotic and abiotic stresses in breeding programs. However, the introgression of some beneficial wild traits may be paralleled by negative effects on some other important agronomic and quality traits. From the quality point of view, virgin olive oil (VOO) from olive cultivars is highly appreciated for its fatty acid composition (high monounsaturated oleic acid content) and the presence of several minor components. However, the composition of VOO from wild origin and its comparison with VOO from olive cultivars has been scarcely studied. In this work, the variability for fruit characters (fruit weight and oil content, OC), fatty acid composition, and minor quality components (squalene, sterols and tocopherols content and composition) was studied in a set of plant materials involving three different origins: wild genotypes (n = 32), cultivars (n = 62) and genotypes belonging to cultivar × wild progenies (n = 62). As expected, values for fruit size and OC in wild olives were lower than those obtained in cultivated materials, with intermediate values for cultivar × wild progenies. Wild olives showed a remarkably higher C16:0 percentage and tocopherol content in comparison to the cultivars. Contrarily, lower C18:1 percentage, squalene and sterol content were found in the wild genotypes, while no clear differences were found among the different plant materials regarding composition of the tocopherol and phytosterol fractions. Some common highly significant correlations among components of the same chemical family were found in all groups of plant materials. However, some other correlations were specific for one of the groups. The results of the study suggested that the use of wild germplasm in olive breeding programs will not have a negative impact on fatty acid composition, tocopherol content, and tocopherol and phytosterol profiles provided that selection for these compounds is conducted from early generations. Important traits such as tocopherol content could be even improved by using wild parents.