The Mediterranean diet and olive oil as its quintessential part are almost synonymous with a healthy way of eating and living nowadays. This kind of diet has been highly appreciated and is widely recognized for being associated with many favorable effects, such as reduced incidence of different chronic diseases and prolonged longevity. Although olive oil polyphenols present a minor fraction in the composition of olive oil, they seem to be of great importance when it comes to the health benefits, and interest in their biological and potential therapeutic effects is huge. There is a growing body of in vitro and in vivo studies, as well as intervention-based clinical trials, revealing new aspects of already known and many new, previously unknown activities and health effects of these compounds. This review summarizes recent findings regarding biological activities, metabolism and bioavailability of the major olive oil phenolic compounds—hydroxytyrosol, tyrosol, oleuropein, oleocanthal and oleacein—the most important being their antiatherogenic, cardioprotective, anticancer, neuroprotective and endocrine effects. The evidence presented in the review concludes that these phenolic compounds have great pharmacological potential, however, further studies are still required.