Atomic physics experiments commonly use millitesla-scale magnetic fields to provide a quantization axis. As atomic transition frequencies depend on the magnitude of this field, many experiments require a stable absolute field. Most setups use electromagnets, which require a power supply stability not usually met by commercially available units. We demonstrate stabilization of a field of 14.6 mT to 4.3 nT rms noise (0.29 ppm), compared to noise of > 100 nT without any stabilization. The rms noise is measured using a field-dependent hyperfine transition in a single 43 Ca + ion held in a Paul trap at the centre of the magnetic field coils. For the 43 Ca + "atomic clock" qubit transition at 14.6 mT, which depends on the field only in second order, this would yield a projected coherence time of many hours. Our system consists of a feedback loop and a feedforward circuit that control the current through the field coils and could easily be adapted to other field amplitudes, making it suitable for other applications such as neutral atom traps.