Nonlinear data assimilation in geosciences: an extremely efficient particle filter

Leeuwen, Peter Jan van

doi:10.1002/qj.699

Cited by 335 publications

(317 citation statements)

References 21 publications

Supporting

Mentioning

313

Contrasting

Unclassified

Order By: Relevance

“…Our method does not require any special structure for the state dynamics: they may be stochastic or deterministic, all we need is the ability to simulate from the state transitions. This contrasts with some of the recent generalizations to particle filtering as proposed in, e.g., van Leeuwen (2010) or Morzfeld & Chorin (2012).…”

Section: Introductionmentioning

confidence: 65%

Bridging the ensemble Kalman and particle filters

2013

View full text Add to dashboard Cite

SUMMARYIn many applications of Monte Carlo nonlinear filtering, the propagation step is computationally expensive, and hence the sample size is limited. With small sample sizes, the update step becomes crucial. Particle filtering suffers from the well-known problem of sample degeneracy. Ensemble Kalman filtering avoids this, at the expense of treating non-Gaussian features of the forecast distribution incorrectly. Here we introduce a procedure that makes a continuous transition indexed by γ ∈ [0, 1] between the ensemble and the particle filter update. We propose automatic choices of the parameter γ such that the update stays as close as possible to the particle filter update subject to avoiding degeneracy. In various examples, we show that this procedure leads to updates that are able to handle non-Gaussian features of the forecast sample even in high-dimensional situations.

show abstract

Section: Introductionmentioning

confidence: 65%

Bridging the ensemble Kalman and particle filters

2013

View full text Add to dashboard Cite

show abstract

“…However, the implicit Gaussian assumption leads to a linear update mechanism and renders the analysis suboptimal in nonlinear systems (Lei and Bickel 2011). Consequently, there is broad research activity toward enabling the applicability of nonlinear filters in high dimensions (e.g., van Leeuwen 2009). A more recent development is the equivalent weights PF (EWPF; van Leeuwen 2010).…”

Section: Introductionmentioning

confidence: 99%

“…While the EWPF aims at considering the full analysis pdf, approximations to fully nonlinear filtering have been suggested as well (van Leeuwen 2009). This work builds upon Tödter and Ahrens (2015), who introduced the nonlinear ensemble transform filter (NETF).…”

Section: Introductionmentioning

confidence: 99%

Assessment of a Nonlinear Ensemble Transform Filter for High-Dimensional Data Assimilation

Tödter

Kirchgessner

Nerger

et al. 2016

Monthly Weather Review

View full text Add to dashboard Cite

This work assesses the large-scale applicability of the recently proposed nonlinear ensemble transform filter (NETF) in data assimilation experiments with the NEMO ocean general circulation model. The new filter constitutes a second-order exact approximation to fully nonlinear particle filtering. Thus, it relaxes the Gaussian assumption contained in ensemble Kalman filters. The NETF applies an update step similar to the local ensemble transform Kalman filter (LETKF), which allows for efficient and simple implementation. Here, simulated observations are assimilated into a simplified ocean configuration that exhibits globally highdimensional dynamics with a chaotic mesoscale flow. The model climatology is used to initialize an ensemble of 120 members. The number of observations in each local filter update is of the same order resulting from the use of a realistic oceanic observation scenario. Here, an importance sampling particle filter (PF) would require at least 10 6 members. Despite the relatively small ensemble size, the NETF remains stable and converges to the truth. In this setup, the NETF achieves at least the performance of the LETKF. However, it requires a longer spinup period because the algorithm only relies on the particle weights at the analysis time. These findings show that the NETF can successfully deal with a large-scale assimilation problem in which the local observation dimension is of the same order as the ensemble size. Thus, the second-order exact NETF does not suffer from the PF's curse of dimensionality, even in a deterministic system.

show abstract

“…The equivalent weights particle filter has already been introduced by Van Leeuwen (2010, 2011. It uses a proposal density to guide all particles towards the observations and initial results are very exciting.…”

Section: Introductionmentioning

confidence: 99%

An exploration of the equivalent weights particle filter

Ades

Leeuwen

2012

Quart J Royal Meteoro Soc

Self Cite

106

View full text Add to dashboard Cite

Particle filters are fully nonlinear data assimilation techniques that aim to represent the probability distribution of the model state given the observations (the posterior) by a number of particles. In high-dimensional geophysical applications, the number of particles required by the sequential importance resampling (SIR) particle filter (in order to capture the high-probability region of the posterior) is too large to make them usable. However particle filters can be formulated using proposal densities, which give greater freedom in how particles are sampled and allow for a much smaller number of particles. Here a particle filter is presented which uses the proposal density to ensure that all particles end up in the high-probability region of the posterior probability density function. This gives rise to the possibility of nonlinear data assimilation in large-dimensional systems. The particle filter formulation is compared to the optimal proposal density particle filter and the implicit particle filter, both of which also utilise a proposal density. We show that, when observations are available every time step, both schemes will be degenerate when the number of independent observations is large, unlike the new scheme. The sensitivity of the new scheme to its parameter values is explored theoretically and demonstrated using the Lorenz (1963) model.

show abstract

Nonlinear data assimilation in geosciences: an extremely efficient particle filter

Cited by 335 publications

References 21 publications

Bridging the ensemble Kalman and particle filters

Bridging the ensemble Kalman and particle filters

Assessment of a Nonlinear Ensemble Transform Filter for High-Dimensional Data Assimilation

An exploration of the equivalent weights particle filter

Contact Info

Product

Resources

About