Variational Bayes learning of graphical models with hidden variables

Abstract

Hidden variable graphical models are powerful tools to describe high-dimensional data; they capture dependencies between observed variables by introducing a suitable number of hidden variables. Present methods for learning the dependence structure of hidden variable graphical models are derived from the idea of maximizing penalized likelihood, and hence are associated with the troublesome problem of regularization selection. In this paper, we show that this problem can be successfully circumvented by treating the penalty parameters as random variables and describing the hidden variable graphical models in a Bayesian formulation. An efficient variational Bayes algorithm is further developed to adaptively learn the graphical model as well as the distribution of penalty parameters. Numerical results from both synthetic and real data show that the proposed variational Bayes method yields comparable or better performance than the stability selection based maximum penalized likelihood method, yet it requires several orders of magnitude less computational time.

Visual and lidar-based SLAM by variational bayesian methods

“…As an alternative to MCMC methods, variational Bayesian inference (VBI) [12,85,141,177,189,201] are a family of techniques to approximate intractable posterior distributions arising in machine learning and Bayesian inference. Primarily, they can provide an analytical approximation to the posterior probability p(Z|X).…”

“…Variational methods [12,60,85,141,177,189,201] have recently gained popularity in the context of graphical models. They are a family of techniques for approximating complex joint distributions, for example, the distribution in (4.1), by a simpler variational distribution.…”