Two-dimensional Fermi gases with universal short-range interactions are known to exhibit a quantum anomaly, where a classical scale and conformal invariance is broken by quantum effects at strong coupling. We argue that in a quasi two-dimensional geometry, a conformal window remains at weak interactions. Using degenerate perturbation theory, we verify the conformal symmetry by computing the energy spectrum of mesoscopic particle ensembles in a harmonic trap, which separates into conformal towers formed by so-called primary states and their center-of-mass and breathing-mode excitations, the latter having excitation energies at precisely twice the harmonic oscillator energy. In addition, using Metropolis importance sampling, we compute the hyperradial distribution function of the many-body wavefunctions, which are predicted by the conformal symmetry in closed analytical form. The weakly interacting Fermi gas constitutes a system where the non-relativistic conformal symmetry can be revealed using elementary methods, and our results are testable in current experiments on mesoscopic Fermi gases.