The flower is the reproductive organ of the tea plant, while it is also processed into different kinds of products and thus of great significance to be utilized. In this study, the non-volatile secondary metabolites in the internal and external petals of white, white and pink, and pink tea flowers were studied using a widely targeted metabolomics method with ultra-high liquid chromatography–tandem mass spectrometry (UPLC-MS/MS). A total of 429 metabolites were identified, including 195 flavonoids, 121 phenolic acids, 40 alkaloids, 29 lignans and coumarins, 19 tannins, 17 terpenoids, and 8 other metabolites. The metabolites in the internal and external petals of different colored flowers showed great changes in flavonoids. Most flavonoids and all tannins in the internal petals were higher compared with the external petals. Some phenolic acids were more accumulated in the external petals, while others showed opposite trends. The pink tea flower contained more flavonoids, alkaloids, lignans, coumarins, terpenoids, and tannins compared with white tea flowers. In addition, cyanidin-3-O-glucoside was more accumulated in the external petals of the pink flower, indicating that anthocyanin may be the main reason for the color difference between the pink and white tea flower. The enriched metabolic pathways of different colored flowers were involved in flavonoid biosynthesis, glycine, serine and threonine metabolism, glycerophospholipid metabolism, and phenylpropanoid biosynthesis. The findings of this study broaden the current understanding of non-volatile compound changes in tea plants. It is also helpful to lay a theoretical foundation for integrated applications of tea flowers.