We present the first a priori error analysis of a class of space-discretizations by Hybridizable Discontinuous Galerkin (HDG) methods for the time-dependent Maxwell's equations introduced in Comput. Methods Appl. Mech. Engrg., vol. 396, paper. No. 114969, 27 pages, 2022. The distinctive feature of these discretizations is that they display a discrete version of the Hamiltonian structure of the original Maxwell's equations. This is why they are called "Hamiltonian" HDG methods. Because of this, when combined with symplectic time-marching methods, the resulting methods display an energy that does not drift in time. We provide a single analysis for several of these methods by exploiting the fact that they only differ by the choice of the approximation spaces and the stabilization functions. We also introduce a new way of discretizing the static Maxwell's equations in order to define the initial condition in a manner consistent with our technique of analysis. Finally, we present numerical tests to validate our theoretical orders of convergence and to explore the convergence properties of the method in situations not covered by our analysis.