The production of olive oil is accompanied by the generation of a huge amount of waste and by-products including olive leaves, pomace, and wastewater. The latter represents a relevant environmental issue because they contain certain phytotoxic compounds that may need specific treatments before the expensive disposal. Therefore, reducing waste biomass and valorizing by-products would make olive oil production more sustainable. Here, we explore the biological actions of extracts deriving from olive by-products including olive pomace (OP), olive wastewater (OWW), and olive leaf (OLs) in human colorectal carcinoma HCT8 cells. Interestingly, with the same phenolic concentration, the extract obtained from the OWW showed higher antioxidant ability compared with the extracts derived from OP and OLs. These biological effects may be related to the differential phenolic composition of the extracts, as OWW extract contains the highest amount of hydroxytyrosol and tyrosol that are potent antioxidant compounds. Furthermore, OP extract that contains a higher level of vanillic acid than the other extracts displayed a cytotoxic action at the highest concentration. Together these findings revealed that phenols in the by-product extracts may interfere with signaling molecules that cross-link several intracellular pathways, raising the possibility to use them for beneficial health effects.