Olive oil displays remarkable organoleptic and nutritional features, which turn it into a foodstuff appreciated by consumers, and a basic component of the Mediterranean diet. Indeed, the noticed benefits of including olive oil in the diet have been assigned to the presence of diverse bioactive compounds with different molecular structures. These compounds confer a wide range of biological properties to this food matrix, including the prevention of distinct human diseases as well as the modulation of their severity. The most relevant bioactive compounds present in olive oil correspond to benzoic and cinnamic acids, phenolic alcohols and secoiridoids, and also flavonoids. Over the last decades, several studies, devoted to gaining a further insight into the relative contribution of the separate groups and individual compounds for their biological activities, have been conducted, providing relevant information on structure–activity relationships. Therefore, this paper critically reviews the health benefits evidenced by distinct phenolic compounds found in olive oils, thus contributing to clarify the relationship between their chemical structures and biological functions, further supporting their interest as essential ingredients of wholesome foods.