We study the dynamical evolution and PT symmetry breaking of the non-Hermitian double Jaynes-Cummings model with PT symmetry. The results indicate that there exist symmetry maintained and breaking phase in this model, and the non-Hermitian terms lead to different behaviors in two phases. In the PT symmetry phase, there exists stable Rabi oscillation caused by the interaction between photons and atoms. At the exceptional point, the system shows critical property that the probabilities in each state over time tend to be equal. The system will undergo a transition from the PT symmetry phase to the PT symmetry breaking phase, as increasing the coupling constant. In the PT symmetry breaking phase, complex eigenvalues appear, and the probability in each state increases monotonically in an exponential form. This study can help us to understand the effect of symmetry on the interaction between photons and atoms in non-Hermitian systems.