Within the minimal supersymmetric standard model (MSSM), the large tan β regime can lead to important modifications in the pattern of CP-violating sources contributing to low energy electric dipole moments (EDMs). In particular, four-fermion CP-violating interactions induced by Higgs exchange should be accounted for alongside the constituent EDMs of quarks and electrons. To this end, we present a comprehensive analysis of three low energy EDM observables -namely the EDMs of thallium, mercury and the neutronat large tan β, in terms of one-and two-loop contributions to the constituent EDMs and four-fermion interactions. We concentrate on the constrained MSSM as well as the MSSM with non-universal Higgs masses, and include the CP-violating phases of µ and A. Our results indicate that the atomic EDMs receive significant corrections from four-fermion operators, especially when Im(A) is the only CP-violating source, whereas the neutron EDM remains relatively insensitive to these effects. As a consequence, in a large portion of the parameter space, one cannot infer a separate bound on the electron EDM via the experimental constraint on the thallium EDM. Furthermore, we find that the electron EDM can be greatly reduced due to the destructive interference of one-and two-loop contributions with the latter being dominated by virtual staus.