Olives (Olea europaea L.) are a significant part of the agroindustry in China. Olive leaves, the most abundant by-products of the olive and olive oil industry, contain bioactive compounds that are beneficial to human health. The purpose of this study was to evaluate the phytochemical profiles and antioxidant capacities of olive leaves from 32 cultivars grown in China. A total of 32 phytochemical compounds were identified using high-performance liquid chromatography–electrospray ionization–tandem mass spectrometry, including 17 flavonoids, five iridoids, two hydroxycinnamic acids, six triterpenic acids, one simple phenol, and one coumarin. Specifically, olive leaves were found to be excellent sources of flavonoids (4.92–18.29 mg/g dw), iridoids (5.75–33.73 mg/g dw), and triterpenic acids (15.72–35.75 mg/g dw), and considerable variations in phytochemical content were detected among the different cultivars. All tested cultivars were classified into three categories according to their oil contents for further comparative phytochemicals assessment. Principal component analysis indicated that the investigated olive cultivars could be distinguished based upon their phytochemical profiles and antioxidant capacities. The olive leaves obtained from the low-oil-content (<16%) cultivars exhibited higher levels of glycosylated flavonoids and iridoids, while those obtained from high-oil-content (>20%) cultivars contained mainly triterpenic acids in their compositions. Correspondingly, the low-oil-content cultivars (OL3, Frantoio selection and OL14, Huaou 5) exhibited the highest ABTS antioxidant activities (758.01 ± 16.54 and 710.64 ± 14.58 mg TE/g dw, respectively), and OL9 (Olea europaea subsp. Cuspidata isolate Yunnan) and OL3 exhibited the highest ferric reducing/antioxidant power assay values (1228.29 ± 23.95 mg TE/g dw and 1099.99 ± 14.30 mg TE/g dw, respectively). The results from this study may be beneficial to the comprehensive evaluation and utilization of bioactive compounds in olive leaves.