We present the geomagnetic field model COV-OBS.x1, covering 1840 to 2020, from which have been derived candidate models for the IGRF-12. Towards the most recent epochs, it is primarily constrained by first differences of observatory annual means and measurements from the Oersted, Champ, and Swarm satellite missions. Stochastic information derived from the temporal spectra of geomagnetic series is used to construct the a priori model covariance matrix that complements the constraint brought by the data. This approach makes it possible the use of a posteriori model errors, for instance, to measure the 'observations' uncertainties in data assimilation schemes for the study of the outer core dynamics. We also present and illustrate a stochastic algorithm designed to forecast the geomagnetic field. The radial field at the outer core surface is advected by core motions governed by an auto-regressive process of order 1. This particular choice is motivated by the slope observed for the power spectral density of geomagnetic series. Accounting for time-correlated model errors (subgrid processes associated with the unresolved magnetic field) is made possible thanks to the use of an augmented state ensemble Kalman filter algorithm. We show that the envelope of forecasts includes the observed secular variation of the geomagnetic field over 5-year intervals, even in the case of rapid changes. In a purpose of testing hypotheses about the core dynamics, this prototype method could be implemented to build the 'state zero' of the ability to forecast the geomagnetic field, by measuring what can be predicted when no deterministic physics is incorporated into the dynamical model.