We lay the foundations of a Morse homology on the space of connections A(P) on a principal G‐bundle over a compact manifold Y, based on a newly defined gauge‐invariant functional scriptJ on A(P). While the critical points of scriptJ correspond to Yang–Mills connections on P, its L2‐gradient gives rise to a novel system of elliptic equations. This contrasts previous approaches to a study of the Yang–Mills functional via a parabolic gradient flow. We carry out the analytical details of our programme in the case of a compact two‐dimensional base manifold Y. We furthermore discuss its relation to the well‐developed parabolic Morse homology over closed surfaces. Finally, an application of our elliptic theory is given to three‐dimensional product manifolds Y=Σ×S1.