We examine a common origin of four-dimensional flavor, CP, and U(1)R symmetries in the context of heterotic string theory with standard embedding. We find that flavor and U(1)R symmetries are unified into the Sp(2h + 2, ℂ) modular symmetries of Calabi-Yau threefolds with h being the number of moduli fields. Together with the $$ {\mathbb{Z}}_2^{\mathrm{CP}} $$
ℤ
2
CP
CP symmetry, they are enhanced to GSp(2h + 2, ℂ) ≃ Sp(2h + 2, ℂ) ⋊ $$ {\mathbb{Z}}_2^{\mathrm{CP}} $$
ℤ
2
CP
generalized symplectic modular symmetry. We exemplify the S3, S4, T′, S9 non-Abelian flavor symmetries on explicit toroidal orbifolds with and without resolutions and ℤ2, S4 flavor symmetries on three-parameter examples of Calabi-Yau threefolds. Thus, non-trivial flavor symmetries appear in not only the exact orbifold limit but also a certain class of Calabi-Yau three-folds. These flavor symmetries are further enlarged to non-Abelian discrete groups by the CP symmetry.