The influence of three cultivars (‘Carmen’, ‘Kordia’ and ‘Regina’) grafted on six rootstocks (Mahaleb, ‘Colt’, ‘Oblacinska’, ‘M × M 14′, ‘Gisela 5′ and ‘Gisela 6′) on the phenolic profile of sweet cherry fruits was studied during a two-year period. All the individual phenolic compounds were detected using high-pressure liquid chromatography with diode-array detection coupled with mass spectrometry (HPLC-DAD-MSn). In all the examined samples, 54 compounds were identified and divided into five phenolic classes: anthocyanins (4 compounds), flavonols (7), flavanols (11), flavanones (4), and hydroxycinnamic acids (28). Anthocyanins (58%) and hydroxycinnamic acids (31%) showed the greatest amounts in all the examined fruit samples. PCA analysis revealed that among the cultivars, ‘Kordia’ showed the highest phenolic content. Regarding rootstocks, the lowest values of the most important phenolic compounds were obtained in fruits from trees grafted onto the seedling rootstock Mahaleb. Among the clonal rootstocks, the vigorous ‘Colt’ and dwarf ‘Gisela 5′ promoted the highest values of the evaluated phenolic compounds in the cultivars ‘Kordia’ and ‘Carmen’, while the dwarf ‘Oblacinska’ and semi-vigorous ‘M × M 14′ induced the highest values in the cultivar ‘Regina’. By evaluating the influence of cultivars and rootstocks on the phenolic content in fruit, it has been proven that the cultivar has the most significant influence. However, the rootstock also influences the content of a large number of phenolic compounds. The selection of an adequate cultivar/rootstock combination can also be a powerful tool for improving the phenolic content in fruits, and consequently the nutritional value of sweet cherry fruits.