Revising the stochastic iterative ensemble smoother

Abstract

Abstract. Ensemble randomized maximum likelihood (EnRML) is an iterative (stochastic) ensemble smoother, used for large and nonlinear inverse problems, such as history matching and data assimilation. Its current formulation is overly complicated and has issues with computational costs, noise, and covariance localization, even causing some practitioners to omit crucial prior information. This paper resolves these difficulties and streamlines the algorithm without changing its output. These simplifications are achieved through the careful treatment of the linearizations and subspaces. For example, it is shown (a) how ensemble linearizations relate to average sensitivity and (b) that the ensemble does not lose rank during updates. The paper also draws significantly on the theory of the (deterministic) iterative ensemble Kalman smoother (IEnKS). Comparative benchmarks are obtained with the Lorenz 96 model with these two smoothers and the ensemble smoother using multiple data assimilation (ES-MDA).

Behavior of the iterative ensemble-based variational method in nonlinear problems

“…Insertingŵ into Eq. (12), we obtain an estimate of x 0 as follows: Evensen, 1996;Evensen and van Leeuwen, 2000), although the whole sequence of observations is referred to in Eq. (21).…”

“…On the other hand, in some geophysical applications, it is difficult to obtain a sufficiently long sequence of observations to allow spin-up. For example, in data assimilation for the interior of the Earth, such as lithospheric plates (e.g., Kano et al, 2015) and the outer core (e.g., Sanchez et al, 2019;Minami et al, 2020), the timescale of the system dynamics is so long that a sufficient length of an observation sequence is not feasible. It is also difficult to use a long sequence of observations in the Earth's magnetosphere where the amount of observations is limited (e.g., Nakano et al, 2008;Godinez et al, 2016).…”

“…Several studies have suggested that estimations in nonlinear problems can be improved by iterative algorithms in which the ensemble is repeatedly updated in each iteration (e.g., Gu and Oliver, 2007;Kalnay and Yang, 2010;Chen and Oliver, 2012;Sakov, 2013, 2014;Raanes et al, 2019). These iterative algorithms can be regarded as a S. Nakano: Ensemble variational method variant of the 4DEnVar method, based on an approximation of the Gauss-Newton method or the Levenberg-Marquardt method.…”

Behavior of the iterative ensemble-based variational method in nonlinear problems

“…The above discussion is valid regardless of the choice of the ensemble {x the gradient averaged over the region where the ensemble members are distributed under a certain assumption (Raanes et al, 2019). Even if the spread is taken to be large at first, convergence would eventually be achieved by reducing the ensemble spread in each iteration as described in the previous section.…”

“…Discussion started: 17 April 2020 c Author(s) 2020. CC BY 4.0 License.Several studies have suggested that estimation in nonlinear problems can be improved by iterative algorithms in which 25 the ensemble is repeatedly updated in each iteration (e.g., Gu and Oliver, 2007;Kalnay and Yang, 2010; Chen and Oliver, 2012; Sakov, 2013, 2014;Raanes et al, 2019). These iterative algorithms can be regarded as a variant of the 4DEnVar method based on an approximation of the Gauss-Newton method or the Levenberg-Marquardt method.…”