On Serial Observation Processing in Localized Ensemble Kalman Filters

Nerger, Lars

doi:10.1175/mwr-d-14-00182.1

Cited by 32 publications

(26 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With respect to the EnVar, we note that computational cost is not a strong function of the number of observations N obs , since the number of iterations is chosen to be independent of N obs . On the other hand, the computational cost of the Canadian EnKF is proportional to N obs (Houtekamer et al ) and the sequential analysis algorithm can become unstable for large observation volumes (Nerger, ). At CMC, data selection procedures are consequently much more severe for the EnKF system than for the EnVar system.…”

Section: Discussionmentioning

confidence: 99%

Using the hybrid gain algorithm to sample data assimilation uncertainty

Houtekamer

Buehner

Chevrotière

2018

Quart J Royal Meteoro Soc

View full text Add to dashboard Cite

At the Canadian Meteorological Centre (CMC), an ensemble variational (EnVar) data assimilation system is used for the global deterministic prediction system and an ensemble Kalman filter (EnKF) is used for the global ensemble prediction system. These two systems are co-developed and co-evolving at the CMC and in this study we explore how to maximize the impact of having two algorithms.Following earlier work at the European Centre for Medium-Range Weather Forecasts (ECMWF), we perform experiments with a pure EnKF and an EnKF system that is recentered on the EnVar solution, as well as with a hybrid gain configuration, in which the EnKF system is recentered on the mean of the EnKF and EnVar analyses. Encouraged by the results of the hybrid gain algorithm, we modify it to leave half of the members unchanged and to recenter the other half on the EnVar analysis. With this multi-analysis approach, we sample the different design decisions made for the EnKF and EnVar and see corresponding improvements, notably for the stratospheric analysis.An evaluation using humidity-sensitive radiance channels shows more mixed results of the hybrid gain and multi-analysis approaches. An investigation of the spread-skill relation showed that the background ensembles were overdispersive for humidity and this issue was resolved by a reduction of the additive error for humidity. This highlights the fact that diagnostic information from two analysis systems can be used to identify where those systems have room for improvement. Finally, for various aspects of data assimilation systems, we weigh the benefits of algorithmic diversity against the corresponding additional development cost.

show abstract

Section: Discussionmentioning

confidence: 99%

Using the hybrid gain algorithm to sample data assimilation uncertainty

Houtekamer

Buehner

Chevrotière

2018

Quart J Royal Meteoro Soc

View full text Add to dashboard Cite

show abstract

“…Nerger (2015) gives a comparison between LETKF and the ensemble square root filter (ESRF) of Whitaker and Hamill (2002), while Tippett et al (2003) indicate that the ESRF is identical to the ensemble adjustment Kalman filter (EAKF) of Anderson (2001) when using serial singleobservation processing.…”

Section: Data Assimilation Methodsmentioning

confidence: 99%

A local particle filter for high-dimensional geophysical systems

Penny

Miyoshi

2016

Nonlin. Processes Geophys.

View full text Add to dashboard Cite

Abstract. A local particle filter (LPF) is introduced that outperforms traditional ensemble Kalman filters in highly nonlinear/non-Gaussian scenarios, both in accuracy and computational cost. The standard sampling importance resampling (SIR) particle filter is augmented with an observation-space localization approach, for which an independent analysis is computed locally at each grid point. The deterministic resampling approach of Kitagawa is adapted for application locally and combined with interpolation of the analysis weights to smooth the transition between neighboring points. Gaussian noise is applied with magnitude equal to the local analysis spread to prevent particle degeneracy while maintaining the estimate of the growing dynamical instabilities. The approach is validated against the local ensemble transform Kalman filter (LETKF) using the 40-variable Lorenz-96 (L96) model. The results show that (1) the accuracy of LPF surpasses LETKF as the forecast length increases (thus increasing the degree of nonlinearity), (2) the cost of LPF is significantly lower than LETKF as the ensemble size increases, and (3) LPF prevents filter divergence experienced by LETKF in cases with non-Gaussian observation error distributions.

show abstract

“…This can make it difficult to interpret results of sensitivity experiments. In some cases, the combination of sequential processing and localization can make the analysis process unstable (Nerger 2015).…”

Section: Centermentioning

confidence: 99%

Review of the Ensemble Kalman Filter for Atmospheric Data Assimilation

Houtekamer

Zhang

2016

Monthly Weather Review

506

424

View full text Add to dashboard Cite

This paper reviews the development of the ensemble Kalman filter (EnKF) for atmospheric data assimilation. Particular attention is devoted to recent advances and current challenges. The distinguishing properties of three well-established variations of the EnKF algorithm are first discussed. Given the limited size of the ensemble and the unavoidable existence of errors whose origin is unknown (i.e., system error), various approaches to localizing the impact of observations and to accounting for these errors have been proposed. However, challenges remain; for example, with regard to localization of multiscale phenomena (both in time and space). For the EnKF in general, but higher-resolution applications in particular, it is desirable to use a short assimilation window. This motivates a focus on approaches for maintaining balance during the EnKF update. Also discussed are limited-area EnKF systems, in particular with regard to the assimilation of radar data and applications to tracking severe storms and tropical cyclones. It seems that relatively less attention has been paid to optimizing EnKF assimilation of satellite radiance observations, the growing volume of which has been instrumental in improving global weather predictions. There is also a tendency at various centers to investigate and implement hybrid systems that take advantage of both the ensemble and the variational data assimilation approaches; this poses additional challenges and it is not clear how it will evolve. It is concluded that, despite more than 10 years of operational experience, there are still many unresolved issues that could benefit from further research. Contents Introduction...4490 Popular flavors of the EnKF algorithm...4491 General description...4491 Stochastic and deterministic filters...4492 The stochastic filter...4492 The deterministic filter...4492 Sequential or local filters...4493 Sequential ensemble Kalman filters...4493 The local ensemble transform Kalman filter...4494 Extended state vector...4494 Issues for the development of algorithms...4495 Use of small ensembles...4495 Monte Carlo methods...4495 Validation of reliability...4497 Use of group filters with no inbreeding...4498 Sampling error due to limited ensemble size: The rank problem...4498 Covariance localization...4499 Localization in the sequential filter...4499 Localization in the LETKF...4499 Issues with localization...4500 Summary...4501 Methods to increase ensemble spread...4501 Covariance inflation...4501 Additive inflation...4501 Multiplicative inflation...4502 Relaxation to prior ensemble information...4502 Issues with inflation...4503 Diffusion and truncation...4503 Error in physical parameterizations...4504 Physical tendency perturbations...4504 Multimodel, multiphysics, and multiparameter approaches...4505 Future directions...4505 Realism of error sources...4506 Balance and length of the assimilation window...4506 The need for balancing methods...4506 Time-filtering methods...4506 Toward shorter assimilation windows...4507 Reduction of sources of imbalance...4507 Regional data assimilation...4508 Boundary conditions and consistency across multiple domains...4509 Initialization of the starting ensemble...4510 Preprocessing steps for radar observations...4510 Use of radar observations for convective-scale analyses...4511 Use of radar observations for tropical cyclone analyses...4511 Other issues with respect to LAM data assimilation...4511 The assimilation of satellite observations...4512 Covariance localization...4512 Data density...4513 Bias-correction procedures...4513 Impact of covariance cycling...4514 Assumptions regarding observational error...4514 Recommendations regarding satellite observations...4515 Computational aspects...4515 Parameters with an impact on quality...4515 Overview of current parallel algorithms...4516 Evolution of computer architecture...4516 Practical issues...4517 Approaching the gray zone...4518 Summary...4518 Hybrids with variational and EnKF components...4519 Hybrid background error covariances...4519 E4DVar with the α control variable...4519 Not using linearized models with 4DEnVar...4520 The hybrid gain algorithm...4521 Open issues and recommendations...4521 Summary and discussion...4521 Stochastic or deterministic filters...4522 The nature of system error...4522 Going beyond the synoptic scales...4522 Satellite observations...4523 Hybrid systems...4523 Future of the EnKF...4523 APPENDIX A...4524 Types of Filter Divergence...4524 Classical filter divergence...4524 Catastrophic filter divergence...4524 APPENDIX B...4524 Systems Available for Download...4524 References...4525

show abstract

On Serial Observation Processing in Localized Ensemble Kalman Filters

Cited by 32 publications

References 37 publications

Using the hybrid gain algorithm to sample data assimilation uncertainty

Using the hybrid gain algorithm to sample data assimilation uncertainty

A local particle filter for high-dimensional geophysical systems

Review of the Ensemble Kalman Filter for Atmospheric Data Assimilation

Contact Info

Product

Resources

About