With the rising global burden of inflammatory bowel disease (IBD) and the rising costs of novel biological drugs, there is an increasing need for dietary approaches and functional foods that could modulate the course of IBD. The Mediterranean diet has proven to be efficacious in managing chronic inflammatory diseases, and recent studies have also shown its benefits in the setting of IBD. Since olive oil and its compounds have been shown to provide a considerable anti-inflammatory effect, in this review, we aim to discuss the latest evidence concerning the impact of olive oil and its bioactive compounds on IBD. Numerous preclinical studies have exhibited solid evidence on the mechanisms by which polyphenol-rich extra-virgin olive oil (EVOO) or specific polyphenols like hydroxytyrosol (HT) provide their anti-inflammatory, antioxidative, antitumour, and microbiota-modulation effects. Accordingly, several human studies that explored the effects of olive oil on patients with IBD further confirmed the evidence brought forward by preclinical studies. Nevertheless, there is a need for larger-scale, multicentric, randomized control trials that would finally elucidate olive oil’s level of efficacy in modulating the course of IBD.