Variational Inference MPC using Tsallis Divergence

Abstract

In this paper, we provide a generalized framework for Variational Inference-Stochastic Optimal Control by using the non-extensive Tsallis divergence. By incorporating the deformed exponential function into the optimality likelihood function, a novel Tsallis Variational Inference-Model Predictive Control algorithm is derived, which includes prior works such as Variational Inference-Model Predictive Control, Model Predictive Path Integral Control, Cross Entropy Method, and Stein Variational Inference Model Predictive Control as special cases. The proposed algorithm allows for effective control of the cost/reward transform and is characterized by superior performance in terms of mean and variance reduction of the associated cost. The aforementioned features are supported by a theoretical and numerical analysis on the level of risk sensitivity of the proposed algorithm as well as simulation experiments on 5 different robotic systems with 3 different policy parameterizations.

Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations

“…The first toy example is the Gaussian mixture example from Wang et al (2018). In this example, the true distribution is a finite mixture of Gaussians (MOG)…”

Decision-Making with Auto-Encoding Variational Bayes

“…Better performance perhaps could be achieved with a family of divergences that interpolates between forward KL and χ 2 , such as the Rènyi divergence family [47]. Tail adaptive algorithms [48] might help to select the best performing Rènyi divergence.…”

f-Divergence Variational Inference

“…As the root divergence of the Rényi's α-divergence, χ-divergence and many other useful divergences [17,18], f -divergence is a more inclusive statistical divergence (family) and was utilized to improve the statistical properties [19,20], sharpness [10,21], and surely the generality of variational bounds [10,21,22]. However, most of these works only dealt with some portions of f -divergences for their favorable statistical properties, e.g.…”

“…However, most of these works only dealt with some portions of f -divergences for their favorable statistical properties, e.g. mass-covering [19] and tail-adaptive [20], and did not develop a systematic VI framework that harbors all f -divergences. Meanwhile, since i) the regular f -divergence does not explicitly induce an f -variational bound as elegant as the ELBO [11], χ upper bound (CUBO) [3], or Rényi variational bound (RVB) [2], and ii) only specific choices of f -divergence result in an f -variational bound that trivially depends on the evidence [12], a thorough and comprehensive analysis on the f -divergence VI has been due for a long time.…”

“…We hope this effort can be the last brick to complete the building of f -divergence VI and motivate more useful and efficient VI algorithms in the future. After reviewing the f -divergence and introducing a crafty surrogate f -divergence that is interchangeable with the regular f -divergence, we make the following contributions: c1) We enrich the f -divergence VI theory by introducing an f -VI scheme via minimizing a surrogate f -divergence, which makes our f -VI framework compatible with the traditional VI approaches and naturally unifies an amount of existing VI methods, such as KL-VI [1], α-VI [2], χ-VI [3], and their related developments [7][8][9][10]20]. c2) We derive an f -variational bound for the evidence and equip it with the upper/lower bound criteria and an importance-weighted (IW-)bound.…”