An observational program focused on the high redshift (2<z<6) Universe has the opportunity to dramatically improve over upcoming LSS and CMB surveys on measurements of both the standard cosmological model and its extensions. Using a Fisher matrix formalism that builds upon recent advances in Lagrangian perturbation theory, we forecast constraints for future spectroscopic and 21-cm surveys on the standard cosmological model, curvature, neutrino mass, relativistic species, primordial features, primordial non-Gaussianity, dynamical dark energy, and gravitational slip. We compare these constraints with those achievable by current or near-future surveys such as DESI and Euclid, all under the same forecasting formalism, and compare our formalism with traditional linear methods. Our Python code FishLSS — used to calculate the Fisher information of the full shape power spectrum, CMB lensing, the cross-correlation of CMB lensing with galaxies, and combinations thereof — is publicly available.