Electron correlation effects play a key role in stabilising two-electron atoms near the critical nuclear charge, representing the smallest charge required to bind two electrons. However, deciphering the importance of these effects relies on fully understanding the uncorrelated Hartree-Fock description. Here, we investigate the properties of the ground state wave function in the small nuclear charge limit using various symmetry-restricted Hartree-Fock formalisms. We identify the nuclear charge where spin-symmetry breaking occurs to give an unrestricted wave function that predicts the ionisation of an electron. We also discover another critical nuclear charge where the closed-shell electron density detaches from the nucleus, and identify the importance of fractional spin errors and static correlation in this limit.