We provide a consistent and complete calculation of the electric dipole moments of the deuteron, helion, and triton in the framework of chiral effective field theory. The CP-conserving and CP-violating interactions are treated on equal footing and we consider CP-violating one-, two-, and three-nucleon operators up to next-to-leading-order in the chiral power counting. In particular, we calculate for the first time EDM contributions induced by the CP-violating three-pion operator. We find that effects of CP-violating nucleon-nucleon contact interactions are larger than those found in previous studies based on phenomenological models for the CP-conserving nucleon-nucleon interactions. Our results which apply to any model of CP violation in the hadronic sector can be used to test various scenarios of CP violation. As examples, we study the implications of our results on the QCD θ-term and the minimal left-right symmetric model.