Exact Bayesian inference by symbolic disintegration

ShanChung-chieh,; RamseyNorman,

doi:10.1145/3093333.3009852

Cited by 7 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, a method node in a pipeline can be related to a monad aggregating several distributions into a single output distribution (Ścibior et al, 2015). An important challenge in PP is the automation of computing conditional distributions (Shan and Ramsey, 2017). Numerical disintegration and extensions thereof might be of independent interest to this field (e.g.…”

Section: Consistent Estimationmentioning

confidence: 99%

Bayesian Probabilistic Numerical Methods

Cockayne¹,

Oates²,

Sullivan³

et al. 2019

SIAM Rev.

120

123

View full text Add to dashboard Cite

The emergent field of probabilistic numerics has thus far lacked clear statistical principals. This paper establishes Bayesian probabilistic numerical methods as those which can be cast as solutions to certain inverse problems within the Bayesian framework. This allows us to establish general conditions under which Bayesian probabilistic numerical methods are well-defined, encompassing both non-linear and non-Gaussian models. For general computation, a numerical approximation scheme is proposed and its asymptotic convergence established. The theoretical development is then extended to pipelines of computation, wherein probabilistic numerical methods are composed to solve more challenging numerical tasks. The contribution highlights an important research frontier at the interface of numerical analysis and uncertainty quantification, with a challenging industrial application presented.

show abstract

Section: Consistent Estimationmentioning

confidence: 99%

Bayesian Probabilistic Numerical Methods

Cockayne¹,

Oates²,

Sullivan³

et al. 2019

SIAM Rev.

120

123

View full text Add to dashboard Cite

show abstract

“…More generally, disintegration and Bayesian inversion are used to structurally organise state updates in the presence of new evidence in probabilistic programming; see e.g., Borgström et al (2013), Gordon et al (2014), Katoen et al (2015) and Staton et al (2016). See also Shan and Ramsey (2017), where disintegration is handled via symbolic manipulation.…”

Section: Introductionmentioning

confidence: 99%

Disintegration and Bayesian inversion via string diagrams

Cho

Jacobs

2019

Math. Struct. Comp. Sci.

125

View full text Add to dashboard Cite

The notions of disintegration and Bayesian inversion are fundamental in conditional probability theory. They produce channels, as conditional probabilities, from a joint state, or from an already given channel (in opposite direction). These notions exist in the literature, in concrete situations, but are presented here in abstract graphical formulations. The resulting abstract descriptions are used for proving basic results in conditional probability theory. The existence of disintegration and Bayesian inversion is discussed for discrete probability, and also for measure-theoretic probability – via standard Borel spaces and via likelihoods. Finally, the usefulness of disintegration and Bayesian inversion is illustrated in several examples.

show abstract

“…Narayanan et al [47] and Zinkov and Shan [63] validated the soundness of program transformations in Hakaru, which contains a programmable MH algorithm. The development of Hakaru is not centered around sample traces, and it uses symbolic disintegration [15,54] to calculate the marginal densities for computing the acceptance ratio in an MH step. In this paper, we focus on a trace-based scheme for programmable inference.…”

Section: Related Workmentioning

confidence: 99%

Sound probabilistic inference via guide types

Wang

Hoffmann

Reps³

2021

Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

Probabilistic programming languages aim to describe and automate Bayesian modeling and inference. Modern languages support programmable inference, which allows users to customize inference algorithms by incorporating guide programs to improve inference performance. For Bayesian inference to be sound, guide programs must be compatible with model programs. One pervasive but challenging condition for model-guide compatibility is absolute continuity, which requires that the model and guide programs define probability distributions with the same support.This paper presents a new probabilistic programming language that guarantees absolute continuity, and features general programming constructs, such as branching and recursion. Model and guide programs are implemented as coroutines that communicate with each other to synchronize the set of random variables they sample during their execution. Novel guide types describe and enforce communication protocols between coroutines. If the model and guide are well-typed using the same protocol, then they are guaranteed to enjoy absolute continuity. An efficient algorithm infers guide types from code so that users do not have to specify the types. The new programming language is evaluated with an implementation that includes the type-inference algorithm and a prototype compiler that targets Pyro. Experiments show that our language is capable of expressing a variety of probabilistic models with nontrivial control flow and recursion, and that the coroutine-based computation does not introduce significant overhead in actual Bayesian inference.

show abstract

Exact Bayesian inference by symbolic disintegration

Cited by 7 publications

References 42 publications

Bayesian Probabilistic Numerical Methods

Bayesian Probabilistic Numerical Methods

Disintegration and Bayesian inversion via string diagrams

Sound probabilistic inference via guide types

Contact Info

Product

Resources

About