Review article: Comparison of local particle filters and new implementations

Abstract

Abstract. Particle filtering is a generic weighted ensemble data assimilation method based on sequential importance sampling, suited for nonlinear and non-Gaussian filtering problems. Unless the number of ensemble members scales exponentially with the problem size, particle filter (PF) algorithms experience weight degeneracy. This phenomenon is a manifestation of the curse of dimensionality that prevents the use of PF methods for high-dimensional data assimilation. The use of local analyses to counteract the curse of dimensionality was suggested early in the development of PF algorithms. However, implementing localisation in the PF is a challenge, because there is no simple and yet consistent way of gluing together locally updated particles across domains. In this article, we review the ideas related to localisation and the PF in the geosciences. We introduce a generic and theoretical classification of local particle filter (LPF) algorithms, with an emphasis on the advantages and drawbacks of each category. Alongside the classification, we suggest practical solutions to the difficulties of local particle filtering, which lead to new implementations and improvements in the design of LPF algorithms. The LPF algorithms are systematically tested and compared using twin experiments with the one-dimensional Lorenz 40-variables model and with a two-dimensional barotropic vorticity model. The results illustrate the advantages of using the optimal transport theory to design the local analysis. With reasonable ensemble sizes, the best LPF algorithms yield data assimilation scores comparable to those of typical ensemble Kalman filter algorithms, even for a mildly nonlinear system.

A localized weighted ensemble Kalman filter for high‐dimensional systems

“…Some of the histograms have high values at both ends, which means that the truth falls outside the range of the particles. This also occurred to most PFs (Farchi and Bocquet, 2018). Note that all the histograms are stacked more or less to the left due to model errors, while the histograms of the LWEnKF appear to be the most balanced, which reflects the ability of the new algorithm to handle model errors.…”

“…Generally, is slightly less than 1. There are other better-performing methods to avoid filter degeneracy, such as those described by Farchi and Bocquet (2018) and Poterjoy et al (2019). Considering that the focus of this paper is on how to localize the WEnKF, we will adopt this simple adjustment.…”

“…In addition, similar to Poterjoy (2016), a non-parametric probability mapping approach called kernel density distribution mapping (KDDM; McGinnis et al 2015) is used after all the observations at time step n are assimilated sequentially. As explained by Poterjoy (2016), Farchi and Bocquet (2018) and Poterjoy et al (2019), the posterior particles are adjusted by KDDM to have the same quantile as the proposal particles. Experiments will test the effectiveness of this approach in the LWEnKF.…”

“…When the case is a high-dimensional model with a large number of independent observations, as assimilation progresses forward in time, the weights of a few particles will become increasingly large, and the weights of most particles will be very small: near (or equal to) zero. This occurs even in simple models (Farchi and Bocquet, 2018). Thus, only a small number of particles (occasionally only one particle) can contribute to characterizing the PDF, while most of them are useless.…”

“…The idea of calculating the particle weights locally was first introduced by Bengtsson et al (2003) and van Leeuwen (2003) and was discussed in detail in van Leeuwen (2009). Farchi and Bocquet (2018) divided the localization of PFs into two categories: state-block domain localization and sequential observation localization. The idea of state-block domain localization is similar to that of the LETKF.…”