Relational Reasoning for Markov Chains in a Probabilistic Guarded Lambda Calculus

Abstract

We extend the simply-typed guarded λ-calculus with discrete probabilities and endow it with a program logic for reasoning about relational properties of guarded probabilistic computations. This provides a framework for programming and reasoning about infinite stochastic processes like Markov chains. We demonstrate the logic sound by interpreting its judgements in the topos of trees and by using probabilistic couplings for the semantics of relational assertions over distributions on discrete types. The program logic is designed to support syntax-directed proofs in the style of relational refinement types, but retains the expressiveness of higher-order logic extended with discrete distributions, and the ability to reason relationally about expressions that have different types or syntactic structure. In addition, our proof system leverages a well-known theorem from the coupling literature to justify better proof rules for relational reasoning about probabilistic expressions. We illustrate these benefits with a broad range of examples that were beyond the scope of previous systems, including shift couplings and lump couplings between random walks.The goal of this paper is to develop a programming and reasoning framework for probabilistic computations over infinite objects, such as Markov chains. Although programming and reasoning frameworks for infinite objects and probabilistic computations are well-understood in isolation, their combination is challenging. In particular, one must develop a proof system that is powerful enough for proving interesting properties of probabilistic computations over infinite objects, and practical enough to support effective verification of these properties.Modelling probabilistic infinite objects A first challenge is to model probabilistic infinite objects. We focus on the case of Markov chains, due to its importance. A (discrete-time) Markov chain is a sequence of random variables {X i } over some fixed type T satisfying some independence property. Thus, the straightforward way of modelling a Markov chain is as a stream of distributions over T . Going back to the simple example outlined above, it is natural to think about this kind of discrete-time Markov chain as characterized by the sequence of positions {p i } i∈N , which in turn can be described as an infinite set indexed by the natural numbers. This suggests that a natural way to model such a Markov chain is to use streams in which each element is produced probabilistically from the previous one. However, there are some downsides to this representation. First of all, it requires explicit reasoning about probabilistic dependency, since X i+1 depends on X i . Also, we might be interested in global properties of the executions of the Markov chain, such as "The probability of passing through the initial state infinitely many times is 1". These properties are naturally expressed as properties of the whole stream. For these reasons, we want to represent Markov chains as distributions over streams. Seemingly, one downside of this repre...

A separation logic for concurrent randomized programs

“…Others may have drawn a number where far more than the first k bits will be 0: these threads would have performed an increment even if the value in the counter were larger than k. The scheduler can exploit this fact to maximize the value of the counter by running each thread one after the other in order of how many 0 bits they have at the beginning of their random number. 1 In Figure 1c we present a new concurrent version that is statistically unbiased, yet retains the same good properties of low contention. 2 Our increment function reads the current value in the counter, then takes the minimum of that value and a parameter MAX.…”

Formal verification of higher-order probabilistic programs: reasoning about approximation, convergence, Bayesian inference, and optimization

“…We show this in a parametrized way by using PPV to define a family of graded ⊤⊤-liftings, a logical relation-like technique to construct predicates/relations over probability distributions, starting from predicates/relations over values. As a concrete application, we embed two recent probabilistic logics: a union bound logic for reasoning about accuracy [Barthe 38:4 T. Sato, A. Aguirre, G. Barthe, M. Gaboardi, D. Garg and J. Hsu et al 2016b], and a logic for reasoning about probability distributions through couplings [Aguirre et al 2018].…”

“…We extend this approach to higherorder probabilistic programs with continuous random variables. A similar approach has been used for discrete random variables by Aguirre et al [2018] in order to reason about unary and relational properties of Markov chains. Our contribution differs significantly from the one by Aguirre et al [2018].…”

“…A similar approach has been used for discrete random variables by Aguirre et al [2018] in order to reason about unary and relational properties of Markov chains. Our contribution differs significantly from the one by Aguirre et al [2018]. Assertions in their framework are non-probabilistic and are interpreted first over deterministic values, and then over distributions over values by probabilistic lifting.…”

Monadic refinements for relational cost analysis

“…However, this limitation does not seem to be fundamental. In recent work [Aguirre et al 2017a], a subset of the authors of this paper (and others) have re-worked a version of RHOL/UHOL based on the guarded λ-calculus [Clouston et al 2016] and a model in the topos of trees, which generalizes sets. This version supports infinite computations, including computations over infinite streams, and allows proving properties of all finite prefixes of the computations.…”