We present a new lattice Monte Carlo approach developed for studying large numbers of strongly interacting nonrelativistic fermions and apply it to a dilute gas of unitary fermions confined to a harmonic trap. In place of importance sampling, our approach makes use of high statistics, an improved action, and recently proposed statistical techniques. We show how improvement of the lattice action can remove discretization and finite volume errors systematically. For N = 3 unitary fermions in a box, our errors in the energy scale as the inverse lattice volume, and we reproduce a previous high precision benchmark calculation to within our 0.3% uncertainty; as additional benchmarks we reproduce precision calculations of N = 3, ..., 6 unitary fermions in a harmonic trap to within our ∼ 1% uncertainty. We then use this action to determine the ground state energies of up to 70 unpolarized fermions trapped in a harmonic potential on a lattice as large as 64 3 × 72. In contrast to variational calculations we find evidence for persistent deviations from the thermodynamic limit for the range of N considered.