Variationally Inferred Sampling through a Refined Bound

Abstract

In this work, a framework to boost the efficiency of Bayesian inference in probabilistic models is introduced by embedding a Markov chain sampler within a variational posterior approximation. We call this framework “refined variational approximation”. Its strengths are its ease of implementation and the automatic tuning of sampler parameters, leading to a faster mixing time through automatic differentiation. Several strategies to approximate evidence lower bound (ELBO) computation are also introduced. Its efficient performance is showcased experimentally using state-space models for time-series data, a variational encoder for density estimation and a conditional variational autoencoder as a deep Bayes classifier.

AI in drug development: a multidisciplinary perspective

“…The first one is stochastic-gradient Markov Chain Monte Carlo (SG-MCMC) methods, which use an estimate of the gradient plus some adequately sampled noise to explore the posterior distribution [64][65][66]. On the other hand, variational Bayes approaches approximate the posterior distribution with a simpler, tractable distribution, such as a Gaussian, by solving an optimisation problem to get the best approximation [67][68][69].…”

Variational Gaussian Mixture Models with robust Dirichlet concentration priors for virtual population generation in hypertrophic cardiomyopathy: a comparison study

“…which is referred to as the Evidence Lower Bound Objective (ELBO) [17]. In the case of GMM, where the search model is a multivariate normal distribution, (6) becomes:…”

“…Due to its Bayesian rationale, VI needs more hyperparameters than ΕΜ, the most important of these being the concentration prior of the BGMM [17][18][19]. A common practice is to define the BGMM's prior structure according to a Dirichlet process mixture with concentration (or gamma) values equal to the inverse of the number of the Gaussian components [17][18][19]. However, in that case, a small weight concentration value with many components would make the model put most of the weights close to zero [17][18][19].…”

“…A common practice is to define the BGMM's prior structure according to a Dirichlet process mixture with concentration (or gamma) values equal to the inverse of the number of the Gaussian components [17][18][19]. However, in that case, a small weight concentration value with many components would make the model put most of the weights close to zero [17][18][19]. In this work, we utilize Dirichlet process mixtures as the BGMM's prior structure, where the concentration of each component on the weight distribution is defined as an exponential function of different Gaussian components to yield a stable number of components across multiple runs.…”

An Information-Theoretic Perspective on Proper Quaternion Variational Autoencoders

“…Recent advances on VAEs focus both on theoretical aspects, such as the improvement of the stochastic inference approach [ 22 ], and on architectural aspects, such as the use of different types of latent variables to learn local and global structures [ 23 ], the definition of hierarchical schemes [ 24 ], rather than the use of a multi-agent generator [ 25 ]. Among the most recent VAE models, we focus on the quaternion-valued variational autoencoder (QVAE), which exploits the properties of quaternion algebra to improve performance, on one hand, and to significantly reduce the overall number of network parameters [ 26 ], on the other hand.…”