We discuss families of approximate quantum error correcting codes which arise as the nearly-degenerate ground states of certain quantum many-body Hamiltonians composed of non-commuting terms. For exact codes, the conditions for error correction can be formulated in terms of the vanishing of a two-sided mutual information in a low-temperature thermofield double state. We consider a notion of distance for approximate codes obtained by demanding that this mutual information instead be small, and we evaluate this mutual information for the SYK model and for a family of low-rank SYK models. After an extrapolation to nearly zero temperature, we find that both kinds of models produce fermionic codes with constant rate as the number, N, of fermions goes to infinity. For SYK, the distance scales as N1/2, and for low-rank SYK, the distance can be arbitrarily close to linear scaling, e.g. N.99, while maintaining a constant rate. We also consider an analog of the no low-energy trivial states property which we dub the no low-energy adiabatically accessible states property and show that these models do have low-energy states that can be prepared adiabatically in a time that does not scale with system size N. We discuss a holographic model of these codes in which the large code distance is a consequence of the emergence of a long wormhole geometry in a simple model of quantum gravity.