Advances in experimental techniques make it possible to map the high redshift Universe in three dimensions at high fidelity in the near future. This will increase the observed volume by many-fold, while providing unprecedented access to very large scales, which hold key information about primordial physics. Recently developed theoretical techniques, together with the smaller size of non-linearities at high redshift, allow the reconstruction of an order of magnitude more "primordial modes", and should improve our understanding of the early Universe through measurements of primordial non-Gaussianity and features in the primordial power spectrum. In addition to probing the first epoch of accelerated expansion, such measurements can probe the Dark Energy density in the dark matter domination era, tightly constraining broad classes of dynamical Dark Energy models. The shape of the matter power spectrum itself has the potential to detect sub-percent fractional amounts of Early Dark Energy to z ∼ 10 5 , probing Dark Energy all the way to when the Universe was only a few years old. The precision of these measurements, combined with CMB observations, also has the promise of greatly improving our constraints on the effective number of relativistic species, the masses of neutrinos, the amount of spatial curvature and the gravitational slip. Studies of linear or quasi-linear large-scale structure with redshift surveys and the CMB currently provide our tightest constraints on cosmology and fundamental physics. Pushing the redshift and volume frontier will provide guaranteed, significant improvements in the state-of-the-art in a manner that is easy to forecast and optimize.