Towards verified stochastic variational inference for probabilistic programs

Abstract

Probabilistic programming is the idea of writing models from statistics and machine learning using program notations and reasoning about these models using generic inference engines. Recently its combination with deep learning has been explored intensely, which led to the development of so called deep probabilistic programming languages, such as Pyro, Edward and ProbTorch. At the core of this development lie inference engines based on stochastic variational inference algorithms. When asked to find information about the posterior distribution of a model written in such a language, these algorithms convert this posterior-inference query into an optimisation problem and solve it approximately by a form of gradient ascent or descent. In this paper, we analyse one of the most fundamental and versatile variational inference algorithms, called score estimator or REINFORCE, using tools from denotational semantics and program analysis. We formally express what this algorithm does on models denoted by programs, and expose implicit assumptions made by the algorithm on the models. The violation of these assumptions may lead to an undefined optimisation objective or the loss of convergence guarantee of the optimisation process. We then describe rules for proving these assumptions, which can be automated by static program analyses. Some of our rules use nontrivial facts from continuous mathematics, and let us replace requirements about integrals in the assumptions, such as integrability of functions defined in terms of programs' denotations, by conditions involving differentiation or boundedness, which are much easier to prove automatically (and manually). Following our general methodology, we have developed a static program analysis for the Pyro programming language that aims at discharging the assumption about what we call model-guide support match. Our analysis is applied to the eight representative model-guide pairs from the Pyro webpage, which include sophisticated neural network models such as AIR. It finds a bug in two of these cases, and shows that the assumptions are met in the others.In this paper, we consider inference engines that lie at the core of so called deep probabilistic programming languages, such as Pyro [Bingham et al. 2019], Edward [Tran et al. 2018, 2016 and ProbTorch [Siddharth et al. 2017]. These languages let their users freely mix deep neural networks with constructs from probabilistic programming, in particular, those for writing Bayesian probabilistic models. In so doing, they facilitate the development of probabilistic deep-network models that may address the problem of measuring the uncertainty in current non-Bayesian deepnetwork models; a non-Bayesian model may predict that the price of energy goes up and that of a house goes down, but it cannot express, for instance, that the model is very confident with the first prediction but not the second.The primary inference engines for these deep probabilistic programming languages implement stochastic variational inference algorithms. Converting infer...

Densities of Almost Surely Terminating Probabilistic Programs are Differentiable Almost Everywhere

“…The notions of weight function and value function in this paper are inspired by the more standard trace-based operational semantics of Borgström et al [8] (see also [52,31]).…”

“…This includes the well-known methods of Hamiltonian Monte-Carlo [15,37] and stochastic variational inference [18,40,6,27]. But these techniques can only be applied when the derivative exists "often enough", and thus, in the context of probabilistic programming, almost everywhere differentiability is often cited as a requirement for correctness [55,31]. The question of which probabilistic programs satisfy this property was selected by Hongseok Yang in his FSCD 2019 invited lecture [54] as one of three open problems in the field of semantics for probabilistic programs.…”

“…In much of the probabilistic programming literature (e.g. [31,55,54], including this paper), the above-mentioned weight function on traces is called the density of the probabilistic program. This may be confusing: as we have seen, a probabilistic program induces a posterior probability distribution on return values, and it is natural to ask whether this distribution admits a probability density function (Radon-Nikodym derivative) w.r.t.…”

Compiling Stan to generative probabilistic languages and extension to deep probabilistic programming

“…Similarly, in Appendix B.2 of [15], Gorinova et al outline the generative translation of Section 2.1, and also mention the issue with multiple updates but do not provide a solution. Lee et al [20] introduce a density-based semantics for Pyro, but this semantics does not handle Stan's non-generative features.…”

Trace types and denotational semantics for sound programmable inference in probabilistic languages

“…Concurrently with this work, Lee et al [2019] have developed a detailed analysis of the semantics of stochastic variational inference with custom variational families that sample from normal distributions. They also introduce an abstract interpretation that can verify model-guide support match, a similar condition to the one we check, for a subset of Pyro programs; their analysis is in some ways more limited than ours (it does not handle user-defined functions, for example), but does reason about tensor shape and broadcasting (which we have not considered here).…”