The Geometry of Model Error

Abstract

This paper investigates the nature of model error in complex deterministic nonlinear systems such as weather forecasting models. Forecasting systems incorporate two components, a forecast model and a data assimilation method. The latter projects a collection of observations of reality into a model state. Key features of model error can be understood in terms of geometric properties of the data projection and a model attracting manifold. Model error can be resolved into two components: a projection error, which can be understood as the model's attractor being in the wrong location given the data projection, and direction error, which can be understood as the trajectories of the model moving in the wrong direction compared to the projection of reality into model space. This investigation introduces some new tools and concepts, including the shadowing filter, causal and noncausal shadow analyses, and various geometric diagnostics. Various properties of forecast errors and model errors are described with reference to low-dimensional systems, like Lorenz's equations; then, an operational weather forecasting system is shown to have the same predicted behavior. The concepts and tools introduced show promise for the diagnosis of model error and the improvement of ensemble forecasting systems.

Changing How Earth System Modeling is Done to Provide More Useful Information for Decision Making, Science, and Society

“…For example, increasing the spatial resolution of a specific atmospheric physics model is justifiable in order to predict atmospheric dynamics more precisely (Shaffrey et al 2009;Palmer 2012) but if its computational requirements restrict the inclusion of other details then the model may be less accurate than had an alternative atmospheric model formulation been adopted to allow other component processes to be represented more accurately. The adequacy of a model structure, including the level of detail, can be assessed by the degree to which predictions can recapture the known (and relevant) dynamics of interest, although such assessments are not in widespread use (Judd et al 2008;Le Bauer et al 2013;Smith et al 2013).…”

“…Fourth, such assessments can be used to help identify the most important reducible sources of uncertainty. Such approaches are being developed for numerical weather prediction models, where the poorest performing model features can be identified (Judd et al 2008).…”

Data assimilation in the laboratory using a rotating annulus experiment

“…Ensemble methods exploit the statistics of a forecast ensemble (Evensen , 1994;van Leeuwen, 2010). Finally, gradient descent filters look for model trajectories that 'shadow' observations using the full nonlinear forecast model (Stemler and Judd, 2009;Judd et al, 2008). Data assimilation is a vibrant area of research, and there is as much need now for new approaches to these problems as there was in the 1960s.…”

Forecast improvement in Lorenz 96 system

“…To ameliorate the effect of this type of divergence, scientists have developed different data assimilation techniques. The problem with the inability to build a perfect model has received much interest in recent years (Orrell et al, 2001;Judd and Smith, 2004;Judd et al, 2008). Comparison of the higher-dimensional truth (atmosphere in this case) and its model is done in the state space of the model -the state of the truth is projected onto that space.…”