Interest in plant compounds has increased, given recent evidence regarding their role in human health due to their pleiotropic effects. For example, plant bioactive compounds present in food products, including polyphenols, are associated with preventive effects in various diseases, such as cancer or inflammation. Breast and colorectal cancers are among the most commonly diagnosed cancers globally. Although appreciable advances have been made in treatments, new therapeutic approaches are still needed. Thus, in this study, up to 28 olive leaf extracts were obtained during different seasons and using different drying temperatures. The influence of these conditions on total polyphenolic content (measured using Folin–Ciocalteu assays), antioxidant activity (using Trolox Equivalent Antioxidant Capacity and Ferric Reducing Ability of Plasma assays) and antiproliferative capacity (using 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide, MTT assays) was tested in breast and colorectal cancer cells. Increased phenolic composition and antioxidant and antiproliferative capacity are noted in the extracts obtained from leaves harvested in autumn, followed by summer, spring and winter. Regarding drying conditions, although there is not a general trend, conditions using the highest temperatures lead to the optimal phenolic content and antioxidant and antiproliferative activities in most cases. These results confirm previously published studies and provide evidence in support of the influence of both harvesting and drying conditions on the biological activity of olive leaf extracts.