We explore scenarios for reionizing the intergalactic medium with low galaxy ionizing photon escape fractions. We combine simulation-based halo-mass dependent escape fractions with an extrapolation of the observed galaxy rest-ultraviolet luminosity functions to solve for the reionization history from z = 20 → 4. We explore the posterior distributions for key unknown quantities, including the limiting halo mass for star-formation, the ionizing photon production efficiency, and a potential contribution from active galactic nuclei (AGN). We marginalize over the allowable parameter space using a Markov Chain Monte Carlo method, finding a solution which satisfies the most model-independent constraints on reionization. Our fiducial model can match observational constraints with an average escape fraction of <5% throughout the bulk of the epoch of reionization if: i ) galaxies form stars down to the atomic cooling limit before reionization and a photosuppression mass of log (M h /M ) ∼ 9 during/after reionization (−13 < M UV,lim < −11); ii ) galaxies become more efficient producers of ionizing photons at higher redshifts and fainter magnitudes, and iii ) there is a significant, but sub-dominant, contribution by AGN at z 7. In this model the faintest galaxies (M UV > −15) dominate the ionizing emissivity, leading to an earlier start to reionization and a smoother evolution of the ionized volume filling fraction than models which assume a single escape fraction at all redshifts and luminosities. The ionizing emissivity from this model is consistent with observations at z=4-5 (and below, when extrapolated), in contrast to some models which assume a single escape fraction. Our predicted ionized volume filling fraction at z = 7 of Q H II = 78% (± 8%) is in modest (∼1-2σ) tension with observations of Lyα emitters at z ∼ 7 and the damping wing analyses of the two known z > 7 quasars, which prefer Q H II ,z=7 ∼ 40-50%.