We theoretically study the quantum interference induced photon blockade phenomenon in atom cavity QED system, where the destructive interference between two different transition pathways prohibits the two-photon excitation. Here, we first explore the single atom cavity QED system via an atom or cavity drive. We show that the cavity-driven case will lead to the quantum interference induced photon blockade under a specific condition, but the atom driven case can't result in such interference induced photon blockade. Then, we investigate the two atoms case, and find that an additional transition pathway appears in the atom-driven case. We show that this additional transition pathway results in the quantum interference induced photon blockade only if the atomic resonant frequency is different from the cavity mode frequency. Moreover, in this case, the condition for realizing the interference induced photon blockade is independent of the system's intrinsic parameters, which can be used to generate antibunched photon source both in weak and strong coupling regimes.