We investigate the question whether Lipatov's high energy effective action is capable of reproducing quark and gluon propagators that resum the interaction with a strong background field within high energy factorization. Such propagators are frequently employed in calculations within the color glass condensate formalism, in particular when considering scattering of a dilute projectile on a dense target nucleus or nucleon. We find that such propagators can be obtained from the high energy effective action, if a special parametrization of the gluonic field is used, first proposed by Lipatov in the original publication on the high energy effective action. The obtained propagators are used to rederive from the high energy effective action the leading order Balitsky, Jalilian-Marian, Iancu, McLerran, Weigert, Leonidov, Kovner evolution equation in covariant gauge. As an aside, our result confirms the definition of the Reggeized gluon as the logarithm of an adjoint Wilson line, proposed in the literature.