State and parameter estimation using Monte Carlo evaluation of path integrals

Quinn, JP

doi:10.1002/qj.690

Cited by 25 publications

(29 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The fact that they appear to be (roughly) equal indicates that this condition may be an invariant property of the dynamics. Indeed, we have observed the same phenomenon by using other approaches (e.g., variational optimization and Markov chain Monte Carlo [49,52]), which suggests these other methods may also benefit from the inclusion of time delays.…”

Section: Discussion and Summarysupporting

confidence: 67%

Using waveform information in nonlinear data assimilation

et al. 2014

View full text Add to dashboard Cite

Information in measurements of a nonlinear dynamical system can be transferred to a quantitative model of the observed system to establish its fixed parameters and unobserved state variables. After this learning period is complete, one may predict the model response to new forces and, when successful, these predictions will match additional observations. This adjustment process encounters problems when the model is nonlinear and chaotic because dynamical instability impedes the transfer of information from the data to the model when the number of measurements at each observation time is insufficient. We discuss the use of information in the waveform of the data, realized through a time delayed collection of measurements, to provide additional stability and accuracy to this search procedure. Several examples are explored, including a few familiar nonlinear dynamical systems and small networks of Colpitts oscillators.

show abstract

Section: Discussion and Summarysupporting

confidence: 67%

Using waveform information in nonlinear data assimilation

et al. 2014

View full text Add to dashboard Cite

show abstract

“…Jazwinski 1970), which may be defined as the problem of identifying the minimum spatiotemporal observational density to efficiently counteract error growth (Quinn and Abarbanel 2010). Observability is a necessary condition for the stability of a DA solution, which is in turn a necessary condition to reduce the state-estimation (and prediction) error (Carrassi et al 2008).…”

Section: Data Assimilationmentioning

confidence: 99%

Scientific Challenges of Convective-Scale Numerical Weather Prediction

Yano

Ziemiański

Cullen

et al. 2018

Bulletin of the American Meteorological Society

122

View full text Add to dashboard Cite

After extensive efforts over the course of a decade, convective-scale weather forecasts with horizontal grid spacings of 1–5 km are now operational at national weather services around the world, accompanied by ensemble prediction systems (EPSs). However, though already operational, the capacity of forecasts for this scale is still to be fully exploited by overcoming the fundamental difficulty in prediction: the fully three-dimensional and turbulent nature of the atmosphere. The prediction of this scale is totally different from that of the synoptic scale (103 km), with slowly evolving semigeostrophic dynamics and relatively long predictability on the order of a few days. Even theoretically, very little is understood about the convective scale compared to our extensive knowledge of the synoptic-scale weather regime as a partial differential equation system, as well as in terms of the fluid mechanics, predictability, uncertainties, and stochasticity. Furthermore, there is a requirement for a drastic modification of data assimilation methodologies, physics (e.g., microphysics), and parameterizations, as well as the numerics for use at the convective scale. We need to focus on more fundamental theoretical issues—the Liouville principle and Bayesian probability for probabilistic forecasts—and more fundamental turbulence research to provide robust numerics for the full variety of turbulent flows. The present essay reviews those basic theoretical challenges as comprehensibly as possible. The breadth of the problems that we face is a challenge in itself: an attempt to reduce these into a single critical agenda should be avoided.

show abstract

“…With the current and growing capacity of computers it is becoming relevant and tractable to begin to explore such approaches to inverse problems in differential equations (Kaipio and Somersalo 2005), even though it is currently not feasible to do so for NWP. There has, however, been some limited study of the Bayesian approach to inverse problems in fluid mechanics using path integral formulations in continuous time as introduced in Apte et al (2007); see Apte et al (2008a,b), Quinn and Abarbanel (2010), and Cotter et al (2011) for further developments. We will build on the algorithmic experience contained in these papers here.…”

Section: Introductionmentioning

confidence: 99%

Evaluating Data Assimilation Algorithms

Law

Stuart

2012

Monthly Weather Review

108

View full text Add to dashboard Cite

Data assimilation leads naturally to a Bayesian formulation in which the posterior probability distribution of the system state, given all the observations on a time window of interest, plays a central conceptual role. The aim of this paper is to use this Bayesian posterior probability distribution as a gold standard against which to evaluate various commonly used data assimilation algorithms.A key aspect of geophysical data assimilation is the high dimensionality and limited predictability of the computational model. This paper examines the two-dimensional Navier-Stokes equations in a periodic geometry, which has these features and yet is tractable for explicit and accurate computation of the posterior distribution by state-of-the-art statistical sampling techniques. The commonly used algorithms that are evaluated, as quantified by the relative error in reproducing moments of the posterior, are four-dimensional variational data assimilation (4DVAR) and a variety of sequential filtering approximations based on threedimensional variational data assimilation (3DVAR) and on extended and ensemble Kalman filters.The primary conclusions are that, under the assumption of a well-defined posterior probability distribution, (i) with appropriate parameter choices, approximate filters can perform well in reproducing the mean of the desired probability distribution, (ii) they do not perform as well in reproducing the covariance, and (iii) the error is compounded by the need to modify the covariance, in order to induce stability. Thus, filters can be a useful tool in predicting mean behavior but should be viewed with caution as predictors of uncertainty. These conclusions are intrinsic to the algorithms when assumptions underlying them are not valid and will not change if the model complexity is increased.

show abstract

State and parameter estimation using Monte Carlo evaluation of path integrals

Cited by 25 publications

References 48 publications

Using waveform information in nonlinear data assimilation

Using waveform information in nonlinear data assimilation

Scientific Challenges of Convective-Scale Numerical Weather Prediction

Evaluating Data Assimilation Algorithms

Contact Info

Product

Resources

About