We study a flavor model that the quark sector has the S 3 modular symmetry while the lepton sector has the A 4 modular symmetry. Our model leads to characteristic quark mass matrices which are consistent with experimental data of quark masses, mixing angles and the CP violating phase. The lepton sector is also consistent with the experimental data of neutrino oscillations. We also study baryon and lepton number violations in our flavor model.

We present a flavor model with $S_3$ modular invariance in the framework of SU(5) grand unified theory (GUT). The $S_3$ modular forms of weights $2$ and $4$ give the quark and lepton mass matrices with a common complex parameter, the modulus $\tau$. The GUT relation of down-type quarks and charged leptons is imposed by the vacuum expectation value (VEV) of the adjoint 24-dimensional Higgs multiplet in addition to the VEVs of $5$ and $\bar 5$ Higgs multiplets of SU(5). The observed Cabibbo–Kobayashi–Maskawa and Pontecorvo–Maki–Nakagawa–Sakata mixing parameters as well as the mass eigenvalues are reproduced properly. We discuss the leptonic charge–parity phase and the effective mass of the neutrinoless double beta decay with the sum of neutrino masses.

We study the phenomenological implications of the modular symmetry Γ(3) A 4 of lepton flavors facing recent experimental data of neutrino oscillations. The mass matrices of neutrinos and charged leptons are essentially given by fixing the expectation value of modulus τ , which is the only source of modular invariance breaking. We introduce no flavons in contrast with the conventional flavor models with A 4 symmetry. We classify our neutrino models along with the type I seesaw model, the Weinberg operator model and the Dirac neutrino model. In the normal hierarchy of neutrino masses, the seesaw model is available by taking account of recent experimental data of neutrino oscillations and the cosmological bound of sum of neutrino masses. The predicted sin 2 θ 23 is restricted to be larger than 0.54 and δ CP = ±(50 • -180 • ). Since the correlation of sin 2 θ 23 and δ CP is sharp, the prediction is testable in the future. It is remarkable that the effective mass m ee of the neutrinoless double beta decay is around 22 meV while the sum of neutrino masses is predicted to be 145 meV. On the other hand, for the inverted hierarchy of neutrino masses, only the Dirac neutrino model is consistent with the experimental data.

We study a flavor model with A 4 symmetry which originates from S 4 modular group. In S 4 symmetry, Z 2 subgroup can be anomalous, and then S 4 can be violated to A 4 . Starting with a S 4 symmetric Lagrangian at the tree level, the Lagrangian at the quantum level has only A 4 symmetry when Z 2 in S 4 is anomalous. Decomposing S 4 modular forms into A 4 representations, we obtain the modular forms of two singlets, 1 and 1 , in addition to a triplet for Γ 3 A 4 . We propose a new A 4 flavor model of leptons by using those modular forms. We succeed in constructing the viable neutrino mass matrix through the Weinberg operator for both normal hierarchy (NH) and inverted hierarchy (IH) of neutrino masses. Our predictions of the CP violating Dirac phase δ CP and the mixing sin 2 θ 23 depend on the sum of neutrino masses distinctly for NH.

We study the spontaneous CP violation through the stabilization of the modulus τ in modular invariant flavor models. The CP-invariant potential has the minimum only at Re½τ ¼ 0 or 1=2ðmod 1Þ. From this result, we study CP violation in modular invariant flavor models. The physical CP phase is vanishing. The important point for the CP conservation is the T transformation in the modular symmetry. One needs the violation of T symmetry to realize the CP violation.