Approximate Smoothing and Parameter Estimation in High-Dimensional State-Space Models

Abstract

We present approximate algorithms for performing smoothing in a class of high-dimensional state-space models via sequential Monte Carlo methods ('particle filters'). In high dimensions, a prohibitively large number of Monte Carlo samples ('particles'), growing exponentially in the dimension of the state space, is usually required to obtain a useful smoother. Employing blocking approximations, we exploit the spatial ergodicity properties of the model to circumvent this curse of dimensionality. We thus obtain approximate smoothers that can be computed recursively in time and parallel in space. First, we show that the bias of our blocked smoother is bounded uniformly in the time horizon and in the model dimension. We then approximate the blocked smoother with particles and derive the asymptotic variance of idealised versions of our blocked particle smoother to show that variance is no longer adversely effected by the dimension of the model. Finally, we employ our method to successfully perform maximum-likelihood estimation via stochastic gradient-ascent and stochastic expectationmaximisation algorithms in a 100-dimensional state-space model.

On uncertainty quantification in hydrogeology and hydrogeophysics

“…A detailed theoretical analysis of such a scheme has been proposed in [130] where it was shown rigourously that it can overcome the curse of dimensionality. These methods provide asymptotically biased state and parameter estimates, the bias being controlled under suitable regularity assumptions, or consistent estimates whose mean square errors go to zero at a slower rate than the usual 1/N Monte Carlo rate [130,131]. The main idea behind these techniques is to ignore long-range dependencies when performing Bayes updates in a filtering procedure, an idea borrowed from the ensemble Kalman filter literature where it is referred to as localization [4].…”

“…Second, the smoothing and parameter estimation procedures developed in [131] cannot be applied when only forward simulation of (X t ) t≥1 is feasible. Third, while consistent estimates can be obtained by scaling the size of the blocks with N , the resulting rate of convergence is low and new efficient approaches are…”

Spatiotemporal blocking of the bouncy particle sampler for efficient inference in state-space models

“…( 2010 ) have equivalent computational costs. In Finke and Singh ( 2017 ), an approximate localization scheme is proposed for the forward–backward algorithm, including theoretical results that guarantees bounds on the asymptotic variance and bias under models that are sufficiently local. In the landmark paper of Andrieu et al.…”

Spatiotemporal blocking of the bouncy particle sampler for efficient inference in state space models

“…Similar algorithms like the general two-filter smoother of Briers et al [2010] have equivalent computational costs. In Finke and Singh [2017], an approximate localization scheme is proposed for the forward-backward algorithm, including theoretical results that guarantees bounds on the asymptotic variance and bias under models that are sufficiently local. In the landmark paper of Andrieu et al [2010], the authors introduced particle Markov Chain Monte Carlo, which combines particle filters in conjunction with either Metropolis-Hastings or Gibbs algorithms.…”