The influence of the dephasing effect on the conductance distribution of disordered graphene p-n junctions is studied. Without the dephasing, the conductance distribution has a very wide range and the conductance fluctuation is large. In this case, the conductance plateaus cannot be obtained in a single sample with the fixed disorder configuration. However, by introducing the dephasing, we find that the distribution becomes narrow dramatically and the fluctuation is suppressed strongly, so that the conductance plateaus are obtained clearly for one single sample, which is consistent with experimental measurements. Furthermore, we also investigate the scaling feature of the conductance distribution and find that it has good scaling behavior in the strong dephasing case.