Neural network quantum states are a promising tool to analyze complex quantum systems given their representative power. It can however be difficult to optimize efficiently and effectively the parameters of this type of ansatz. Here we propose a local optimization procedure which, when integrated with stochastic reconfiguration, outperforms previously used global optimization approaches. Specifically, we analyze both the ground state energy and the correlations for the non-integrable tilted Ising model with restricted Boltzmann machines. We find that sequential local updates can lead to faster convergence to states which have energy and correlations closer to those of the ground state, depending on the size of the portion of the neural network which is locally updated. To show the generality of the approach we apply it to both 1D and 2D non-integrable spin systems.