Bayesian system identification of a nonlinear dynamical system using a novel variant of Simulated Annealing

Abstract

a b s t r a c tThis work details the Bayesian identification of a nonlinear dynamical system using a novel MCMC algorithm: 'Data Annealing'. Data Annealing is similar to Simulated Annealing in that it allows the Markov chain to easily clear 'local traps' in the target distribution. To achieve this, training data is fed into the likelihood such that its influence over the posterior is introduced gradually -this allows the annealing procedure to be conducted with reduced computational expense. Additionally, Data Annealing uses a proposal distribution which allows it to conduct a local search accompanied by occasional long jumps, reducing the chance that it will become stuck in local traps. Here it is used to identify an experimental nonlinear system. The resulting Markov chains are used to approximate the covariance matrices of the parameters in a set of competing models before the issue of model selection is tackled using the Deviance Information Criterion.

Bayesian system identification of dynamical systems using large sets of training data: A MCMC solution

“…A possible solution to this problem is to use the Data Annealing algorithm [14]. Noting that, when employing a variant of Simulated Annealing, one is essentially using β to modulate (and increase) the influence of the data on the target distribution, Data Annealing achieves a similar result simply via the gradual introduction of data points into the likelihood.…”

“…} , using MCMC (a standard Metropolis update was employed in [14]). Once a sufficient number of samples have been generated, N can then be increased such that additional data points are included in the likelihood.…”

“…To ensure efficient MCMC performance a proposal density can be chosen whose covariance matrix (assuming a Gaussian proposal is being utilised) is a fraction of the distribution which was most recently targeted. Alternatively, as demonstrated in [14], it is possible to achieve satisfactory results simply by using a heavy-tailed proposal distribution. While Data Annealing tends to be fast (as the model does not have to reproduce the entire set of data every time a sample is generated), one has little control over the rate at which the information in the data is introduced into the likelihood.…”

Bayesian System Identification of MDOF Nonlinear Systems Using Highly Informative Training Data

“…Consequently, the accuracy of the information estimates will depend on the accuracy of one's prior knowledge -this is something which seems intrinsically Bayesian. Throughout this work, with the aim of improving one's prior estimates of θ 0 the Data Annealing (DA) algorithm [6] has been utilised. Essentially, the DA algorithm is similar to the well-known Simulated Annealing algorithm except that, to save computational cost, the annealing procedure is achieved through the gradual introduction of training data into the likelihood.…”

On the Performance of Online Parameter Estimation Algorithms in Systems with Various Identifiability Properties

“…Furthermore, they are very computationally expensive for medium to large dimensional problems. Ongoing research focuses on deriving novel algorithms that more efficiently explore the parameter space [Data Annealing principle, see Green (2015), sampling from a sequence of intermediate distributions which converge to the posterior, see, e.g., Beck and Au (2002), Ching and Chen (2007), and Beck and Zuev (2013)]. …”