In this article we aim at proposing a general mathematical formulation for charge conserving finite element Maxwell solvers coupled with particle schemes. In particular, we identify the finite element continuity equations that must be satisfied by the discrete current sources for several classes of time domain Vlasov-Maxwell simulations to preserve the Gauss law at each time step, and propose a generic algorithm for computing such consistent sources. Since our results cover a wide range of schemes (namely curl-conforming finite element methods of arbitrary degree, general meshes in 2 or 3 dimensions, several classes of time discretization schemes, particles with arbitrary shape factors and piecewise polynomial trajectories of arbitrary degree), we believe that they provide a useful roadmap in the design of high order charge conserving FEM-PIC numerical schemes.