The olive oil industry generates large volumes of by-products, creating notable environmental and economic concerns. Among these, olive cake (OC)—a primary by-product of olive oil extraction—stands out due to its high content of bioactive compounds and potential for value-added recycling. This study focused on characterizing six OC samples from the Trás-os-Montes and Alto Douro regions, collected at different processing times and mills. The samples included two derived from pressing (COC), two from two-phase centrifugation (TPOC; one partially pitted and one dehydrated), and two exhausted OC (EOC) samples. Fundamental analyses assessed total phenols, ortho-diphenols, flavonoids, antioxidant capacity, and tannin content. Results revealed significant variation (p < 0.05) in phenolic composition, namely ortho-diphenols and flavonoid levels among the samples. EOC 2 exhibited the highest concentrations (19.61, 21.82, and 20.12 mg CAT/g, respectively), while COC 2 had the lowest (5.08, 5.08, and 2.76 mg GA/g, respectively). This correlated with elevated antioxidant activity in EOC 2, as measured by FRAP, DPPH, and ABTS assays (129.98, 78.00, and 56.65 μmol Trolox/g). In contrast, COC 1 and COC 2 displayed the lowest antioxidant activities (32.61 μmol Trolox/g in FRAP and 17.24 and 18.98 μmol Trolox/g in DPPH). Tannin analysis showed the highest total tannin content in the dehydrated and pitted OC samples (250.31 and 240.89 mg CAT/100 g), with COC 2 showing the lowest (88.17 mg CAT/100 g). Condensed tannin content varied significantly, with EOC 2 presenting the highest level (328.17 mg CAT/100 g) and COC 2 the lowest one (20.56 mg CAT/100 g). Through HPLC-PDA-MS, 22 compounds were identified, with luteolin and verbascoside being particularly prevalent. This in-depth characterization supports the potential valorization of olive by-products, advancing sustainability and promoting a circular economy in the olive oil sector.