Variational inference and learning for non-linear state-space models with state-dependent observation noise

Abstract

In many real world dynamical systems, the inherent noise levels are not constant but depend on the state. Such aspects are often ignored in modelling because they make inference significantly more complicated. In this paper we propose a variational inference and learning algorithm for a non-linear state-space model with state-dependent observation noise. The observation noise level of each sample depends on additional latent variables with a linear dependence on the latent state. The method yields significant improvements in predictive performance over regular nonlinear state-space model as well as direct autoregressive prediction using Gaussian processes in a simulated Lorenz system with state-dependent noise and in stock price prediction.

Robust Estimation in Non-Linear State-Space Models With State-Dependent Noise

“…stochastic volatility models). With a few exceptions [13], [14], there seems to be very little treatment of SSMs with state-dependent noise in the literature.…”

“…Now that we know 's density we can calculate all of the moments that (12b) requires. In particular, evaluating as defined in (10a) produces 11 (14) Given , the expected value of decreases according to the square of the normalized residual. Intuitively, measurements are down-weighted by their disagreement with the states.…”

“…14 Perhaps the method of Spinello and Stilwell [13] would be a viable candidate, were it not for the fact that their algorithm is designed for filtering and that they assume Gaussian noise. 14 The unscented transform propagates points, while a covariance matrix contains free parameters. (19).…”