The Variational Bayes (VB) approach is used as a one-step approximation for Bayesian filtering. It requires the availability of moments of the free-form distributional optimizers. The latter may have intractable functional forms. In this contribution, we replace these by appropriate fixed-form distributions yielding the required moments. We address two scenarios of this Restricted VB (RVB) approximation. For the first scenario, an application in identification of HMMs is given. In the second, the fixed-form distribution is generated via Particle Filtering (PF). It is shown that a new approximation of Rao-Blackwellized particle filtering is obtained in this scenario of RVB. Its performance is assessed for a simple nonlinear model.
THE VB APPROXIMATIONThe VB approximation is a deterministic free-form optimization technique. It was first used in off-line inference problems [1] and extended to on-line inference of time-invariant parameters in [2]. The use of VB in Bayesian filtering was first discussed in [3]. The key theory is now reviewed.Theorem 1 (Variational Bayes (VB)) Let f (θ|D) be the posterior distribution of multivariate parameter, θ = [θ 1 , θ 2 ] , andf (θ|D) be an approximate distribution restricted to the set of conditionally independent distributions(1)The minimum of the Kullback-Leibler divergencẽis reached forwhere θ /i denotes the complement of θ i in θ. We will refer tõ f (θ|D) (7) as the VB-approximation, andf (θ i |D) (3) as the VB-marginals.The above theorem provides a powerful tool for approximation of joint pdfs in separable form [3]:Here, g (θ 1 , D) and h (θ 2 , D) are finite-dimensional vectors of compatible dimension. Using (4) in (3),where h(·) = Ef (θ2|D) [h (·)] are the VB-moments for θ 2 , and similarly for θ 1 . An Iterative VB (IVB) [3] moment-swapping algorithm is implied. In many non-linear cases of g and/or h, the VB-marginals (3) are non-standard in form, and so the required VB-moments are difficult to evaluate. In this contribution, we aim to replace any such non-standard VB-marginal with a tractable alternative, as follows.Corollary 1 (of Theorem 1: Restricted VB) Letf (θ|D) be a conditionally-independent approximation of f (θ|D) of the kindfwhere f (θ 2 |D) is a fixed-form distribution. Then, the minimal KL divergence (2), under (6), is reached forf (θ 2 |D) needs to be chosen judiciously, such that its momentsrequired in (7)-are available. These moments are substituted just once, without the need for IVB cycles. Some standard distributional approximation methods may be interpreted as special cases of RVB; e.g. (i) certainty equivalence, where f ≡ δ(θ 2 −θ 2 ), in which case (7) becomes the conditional, f θ 1 |D,θ 2 ; and (ii) the Quasi-Bayes (QB) approximation,
BAYESIAN FILTERINGConsider the following model structurewhere θ t is known as the state variable. By Bayesian Filtering (BF), we mean the recursive evaluation of the filtering distribution f (θ t |D t ) using Bayes' rule. D t = [d 1 , . . . , d t ] denotes the history of observations. The computational flowch...
