Chrysanthemi Indici Flos (CIF) is the inflorescence of Chrysanthemum indicum L., which exists in various shades of yellow and has pharmacologically active constituents. It is widely used for medicinal purposes in China, Japan, and South Korea to treat inflammatory diseases. Its external color is usually used to judge its internal quality in trade; however, the correlation between its color and chemical constituents is unknown. Here, we simultaneously determined five phenylpropanoids (neochlorogenic acid, chlorogenic acid, and isochlorogenic acids A, B, and C) and four flavonoids (linarin, luteolin, apigenin, and acacetin) of 70 CIF germplasms using a newly established UPLC method; furthermore, we measured their color parameters (L*, a*, and b*) using a spectrophotometer. Our results showed considerable variations in the bioactive constituent contents and color parameters of CIF. The content of the five phenylpropanoids and the relative correlation degree γi of the nine constituents were positively correlated with color parameters, which could be rapidly predicted based on L* and/or b*. Moreover, we screened out a high-quality germplasm with a high linarin content and bright colors using the hierarchical clustering method. Our results provide comprehensive insight into CIF’s quality evaluation process, particularly the methods for procuring high-quality medicinal materials and breeding by color.