With the development of ultrafast electron and X-ray sources it is becoming possible to study structural dynamics with atomic-level spatial and temporal resolution. Because of their short mean free path, electrons are particularly well suited for investigating surfaces and thin films, such as the challenging and important class of membrane proteins. To perform single-shot diffraction experiments on protein crystals, an ultracold electron source was proposed, based on near-threshold photoionization of laser-cooled atoms, which is capable of producing electron pulses of both high intensity and high coherence. Here we show that high coherence electron pulses can be produced by femtosecond photoionization, opening up a new regime of ultrafast structural dynamics experiments. The transverse coherence turns out to be much better than expected on the basis of the large bandwidth of the femtosecond ionization laser pulses. This surprising result can be explained by analysis of classical electron trajectories.