In this work, the quality parameters of the oils of seven different varieties of olives (Olea europaea cvs, “Arbequina”, “Arbosana”, “Cobrançosa”, “Cornicabra”, “Koroneiki”, “Cacereña”, and “Chiquitita”) grown in hedgerow under cold conditions during three consecutive seasons were analyzed in order to evaluate their adaptation to this growing system. For this purpose, virgin olive oils (VOOs) were extracted, and, in addition to evaluating the indices of hydrolytic, oxidative, and sensory deterioration of the oils, their content in photosynthetic pigments and their composition in fatty acids and phenolic compounds were determined. The correlation between oxidative stability and the parameters evaluated has been studied, with the highest correlation coefficients found for the ratio MUFA/PUFA (0.871) and the content of secoiridoid phenolic compounds (0.816). Furthermore, principal component analysis was performed with the phenolic composition data from each season, which demonstrated the major influence of genetic factors in the phenolic composition of VOO. None of the VOOs presented hydrolytic, oxidative, or sensory deterioration, so all of them remained in the “extra” category. However, it should be noted that the mean ultraviolet absorbance values were significantly higher in “Cornicabra” and significantly lower in “Cacereña” oils. In spite of this, the oxidative stability was significantly higher in “Cornicabra” oils (125 h), which also had the highest mean phenolic content (1035 mg kg−1 oil), while the lowest phenolic content values were found in “Arbequina” and “Chiquitita” (58 and 52 h, respectively).