We solve the ground state of the deuteron using a variational neural network ansatz for the wave function in momentum space. This ansatz provides a flexible representation of both the S and the D states, with relative errors in the energy which are within fractions of a percent of a full diagonalisation benchmark. We extend the previous work on this area in two directions. First, we study new architectures by adding more layers to the network and by exploring different connections between the states. Second, we provide a better estimate of the numerical uncertainty by taking into account the final oscillations at the end of the minimisation process. Overall, we find that the best performing architecture is the simple one-layer, state-independent network. Two-layer networks show indications of overfitting, in regions that are not probed by the fixed momentum basis where calculations are performed. In all cases, the error associated to the model oscillations around the real minimum is larger than the stochastic initialisation uncertainties. The conclusions that we draw can be generalised to other quantum mechanics settings.