A Non-Gaussian Ensemble Filter Update for Data Assimilation

Abstract

A deterministic square root ensemble Kalman filter and a stochastic perturbed observation ensemble Kalman filter are used for data assimilation in both linear and nonlinear single variable dynamical systems. For the linear system, the deterministic filter is simply a method for computing the Kalman filter and is optimal while the stochastic filter has suboptimal performance due to sampling error. For the nonlinear system, the deterministic filter has increasing error as ensemble size increases because all ensemble members but one become tightly clustered. In this case, the stochastic filter performs better for sufficiently large ensembles. A new method for computing ensemble increments in observation space is proposed that does not suffer from the pathological behavior of the deterministic filter while avoiding much of the sampling error of the stochastic filter. This filter uses the order statistics of the prior observation space ensemble to create an approximate continuous prior probability distribution in a fashion analogous to the use of rank histograms for ensemble forecast evaluation. This rank histogram filter can represent non-Gaussian observation space priors and posteriors and is shown to be competitive with existing filters for problems as large as global numerical weather prediction. The ability to represent non-Gaussian distributions is useful for a variety of applications such as convective-scale assimilation and assimilation of bounded quantities such as relative humidity.

Ensemble clustering in deterministic ensemble Kalman filters

“…Our experiments show that S-(L)ETKF tends to shift ensembles away from the Gaussian distribution in nonlinear regimes, consistent with the finding by Anderson (2010). For the SPEEDY model using S-LETKF, this tendency is clear especially for the tracer variables {T, q} in the tropics and the SH, where the sample skewness values are clearly different from zero.…”

“…For the non-linear regime, NS-ETKF has less outliers in the ensemble spread, leading to smaller mean RMSE. This is consistent with the finding by Anderson (2010) that the mean analysis RMSE of the EAKF increased for the larger ensemble size. One can hypothesise if there is any relationship between the mean CD value in the forecast and the analysis RMSE at the end of that window.…”

“…Using the EAKF in highly non-linear scenarios, Anderson (2010) observed ensemble clustering (EC), a phenomenon in which a M-member ensemble splits in an outlier and a tight cluster of MÁ1 members. He found that the EC may occur as a result of the disparity between the non-linear expansion of the ensemble spread in the forecast and the linear contraction of the ensemble spread in the analysis.…”

“…Using other models, this study also showed that the analysis RMSE of the EAKF increased with ensemble size M due to the non-linear expansion in the forecast. To address the EC, Anderson (2010) proposed a rank histogram filter.…”

A non-Gaussian analysis scheme using rank histograms for ensemble data assimilation

“…In this article, it is suggested that the most common remapping methods can only handle weakly non-Gaussian distributions, while the others suffer from sampling issues. In between those two categories, a new remapping method directly applying Bayes' rule, the multivariate rank histogram filter (MRHF), is introduced as an extension of the rank histogram filter (RHF) first introduced by Anderson (2010). Its performance is evaluated and compared with several data assimilation methods, on different levels of non-Gaussianity with the Lorenz 63 model.…”