Nowadays, many individuals, whether healthy or diagnosed with disease, tend to expose themselves to various easily accessible natural products in hopes of benefiting their health and well-being. Mediterranean populations have traditionally used olive oil not only in nutrition but also in cosmetics, including skincare. In this study, the phenolic profile—composed of twelve compounds altogether, including the secoiridoids oleocanthal (OCAL) and oleacein (OCEIN)—of extra virgin olive oil (EVOO) from autochthonous cultivars from Croatia was determined using 1H qNMR spectroscopy and HPLC-DAD analysis, and its biological activity was investigated in melanoma cell lines. The EVOO with the highest OCEIN content had the strongest anti-cancer activity in A375 melanoma cells and the least toxic effect on the non-cancerous keratocyte cell line (HaCaT). On the other hand, pure OCAL was shown to be more effective and safer than pure OCEIN. Post-treatment with any of the EVOO phenolic extracts (EVOO-PEs) enhanced the anti-cancer effect of the anti-cancerous drug dacarbazine (DTIC) applied in pre-treatment, while they did not compromise the viability of non-cancerous cells. The metastatic melanoma A375M cell line was almost unresponsive to the EVOO-PEs themselves, as well as to pure OCEIN and OCAL. Our results demonstrate that olive oils and/or their compounds may have a potentially beneficial effect on melanoma treatment. However, their usage can be detrimental or futile, especially in healthy cells, due to inadequately applied concentrations/combinations or the presence of resistant cells.