Hyperbolic lattices are a new form of synthetic quantum matter in which particles effectively hop on a discrete tessellation of two-dimensional hyperbolic space, a non-Euclidean space of uniform negative curvature. To describe the single-particle eigenstates and eigenenergies for hopping on such a lattice, a hyperbolic generalization of band theory was previously constructed, based on ideas from algebraic geometry. In this hyperbolic band theory, eigenstates are automorphic functions, and the Brillouin zone is a higher-dimensional torus, the Jacobian of the compactified unit cell understood as a higher-genus Riemann surface. Three important questions were left unanswered:(1) whether a band theory can be expected to hold for a non-Euclidean lattice, where translations do not generally commute; (2) whether a formal Bloch theorem can be rigorously established; and (3) whether hyperbolic band theory can describe finite lattices realized in experiment. In the present work, we address all three questions simultaneously. By formulating periodic boundary conditions for finite but arbitrarily large lattices, we show that a generalized Bloch theorem can be rigorously proved, but may or may not involve higher-dimensional irreducible representations (irreps) of the nonabelian translation group, depending on the lattice geometry. Higher-dimensional irreps corrrespond to points in a moduli space of higher-rank stable holomorphic vector bundles, which further generalizes the notion of Brillouin zone beyond the Jacobian. For a large class of finite lattices, only one-dimensional irreps appear, and the hyperbolic band theory previously developed becomes exact.