The influence of semi-hard (C1), hard (C2), and soft whey cheese (C3) immersed in extra virgin olive oil (EVOO) on its oxidative and hydrolytic parameters, fatty acids, and phenolic composition during two months of simultaneous storage was investigated. Accelerated hydrolytic and oxidative degradation was noted in EVOO stored with the immersed cheese compared to control oil. Oxidation indicator (K232), myristic (C 14:0), and trans-oleic fatty acid (C18:1t) exceeded the prescribed limit for the EVOO category in oils stored with immersed C1 and C2, which indicated that standard analytical parameters are ineffective as tools to examine the declared quality and authenticity of such topping oils. The noted changes in fatty acid profile were primarily prescribed to the migration of fats. C1 and C2 influenced a comparable reduction in EVOO total identified phenolic content (−92.1% and −93.5%, respectively), despite having a different content of total proteins and moisture, whereas C3 influenced a slightly lower reduction (−85.0%). Besides the protein profile, other cheese compounds (e.g., moisture, carbohydrates) have been shown to have a considerable role in the development of the EVOO phenolic profile. Finally, compositional changes in EVOO used as a medium for cheese preservation are under significant influence of the cheese’s chemical composition.