Compositional Semantics for Probabilistic Programs with Exact Conditioning

Abstract

We define a probabilistic programming language for Gaussian random variables with a first-class exact conditioning construct. We give operational, denotational and equational semantics for this language, establishing convenient properties like exchangeability of conditions. Conditioning on equality of continuous random variables is nontrivial, as the exact observation may have probability zero; this is Borel's paradox. Using categorical formulations of conditional probability, we show that the good properties of our language are not particular to Gaussians, but can be derived from universal properties, thus generalizing to wider settings. We define the Cond construction, which internalizes conditioning as a morphism, providing general compositional semantics for probabilistic programming with exact conditioning.

Dilations and information flow axioms in categorical probability

“…Recent developments of this framework have resulted in purely categorical proofs of various classical theorems, including theorems on sufficient statistics (Fritz 2020), 0/1-laws (Fritz and Rischel 2020), comparison of statistical experiments , the de Finetti theorem (Fritz et al 2021;Moss and Perrone 2022), development of multinomial and hypergeometric distributions (Jacobs 2021), ergodic systems (Moss and Perrone 2023), and the d-separation criterion for Bayesian networks (Fritz and Klingler 2023). The Markov categories framework has also found use in probabilistic programming theory (Stein 2021;Stein and Staton 2021) and cognitive science (St. Clere Smithe https : //arxiv.org/abs/2109.04461).…”