We present an extension of the canonical coupled mode theory of electromagnetic waves to the case of pulses and spatio-temporal perturbations in complex media. Unlike previous attempts to derive such a model, our approach involves no approximation, and does not impose any restriction on the spatio-temporal profile. Moreover, the effect of modal dispersion on mode evolution and on the coupling to other modes is fully taken into account. Thus, our approach can yield any required accuracy by retaining as many terms in the expansion as needed. It also avoids various artifacts of previous derivations by introducing the correct form of the solution. We then provide what is, to the best of our knowledge, the first ever validation of the coupled mode equations with exact numerical simulations, and demonstrate the wide range of possibilities enabled by spatio-temporal perturbations of pulses, including pulse shortening or broadening or more complex shaping. Our formulation is valid across the electromagnetic spectrum, and can be applied directly also to other wave systems. * Electronic address: sivanyon@bgu.ac.il