Stochastic invariants for probabilistic termination

Abstract

Termination is one of the basic liveness properties, and we study the termination problem for probabilistic programs with real-valued variables. Previous works focused on the qualitative problem that asks whether an input program terminates with probability 1 (almost-sure termination). A powerful approach for this qualitative problem is the notion of ranking supermartingales with respect to a given set of invariants. The quantitative problem (probabilistic termination) asks for bounds on the termination probability, and this problem has not been addressed yet. A fundamental and conceptual drawback of the existing approaches to address probabilistic termination is that even though the supermartingales consider the probabilistic behaviour of the programs, the invariants are obtained completely ignoring the probabilistic aspect (i.e., the invariants are obtained considering all behaviours with no information about the probability).In this work we address the probabilistic termination problem for linear-arithmetic probabilistic programs with nondeterminism. We formally define the notion of stochastic invariants, which are constraints along with a probability bound that the constraints hold. We introduce a concept of repulsing supermartingales. First, we show that repulsing supermartingales can be used to obtain bounds on the probability of the stochastic invariants. Second, we show the effectiveness of repulsing supermartingales in the following three ways: (1) With a combination of ranking and repulsing supermartingales we can compute lower bounds on the probability of termination; (2) repulsing supermartingales provide witnesses for refutation of almost-sure termination; and (3) with a combination of ranking and repulsing supermartingales we can establish persistence properties of probabilistic programs.Along with our conceptual contributions, we establish the following computational results: First, the synthesis of a stochastic invariant which supports some ranking supermartingale and at the same time admits a repulsing supermartingale can be achieved via reduction to the existential first-order theory of reals, which generalizes existing results from the non-probabilistic setting. Second, given a program with "strict invariants" (e.g., obtained via abstract interpretation) and a stochastic invariant, we can check in polynomial time whether there exists a linear repulsing supermartingale w.r.t. the stochastic invariant (via reduction to LP). We also present experimental evaluation of our approach on academic examples.

Ranking and Repulsing Supermartingales for Reachability in Randomized Programs

“…Ranking supermartingales are amenable to template-based synthesis [10,11,13], making them appealing from the automatic analysis point of view. Recently, methods for quantitatively under-approximating reachability probabilities are also proposed in References [15,41].…”

Lexicographic ranking supermartingales: an efficient approach to termination of probabilistic programs

“…The generalization of ranking functions to probabilistic programs is achieved through the ranking supermartingales (RSMs) [15,17,31]. The ranking supermartingales provide a powerful and automated approach for termination analysis of probabilistic programs, and algorithmic approaches for special cases such as linear and polynomial RSMs have also been considered [15,[18][19][20].…”

“…Practical limitations of existing approaches. While an impressive set of theoretical results related to RSMs has been established [15,[17][18][19][20]31], for probabilistic programs with nondeterminism the current approaches are only applicable to academic examples of variants of random walks. The key reason can be understood as follows: even for non-probabilistic programs while ranking functions are sound and complete, they do not necessarily provide a practical approach.…”

Relatively complete verification of probabilistic programs: an expressive language for expectation-based reasoning

“…There is a plethora of methods for proving termination of probabilistic programs based on ranking supermartingales [Chakarov and Sankaranarayanan 2013;Chatterjee et al 2016bChatterjee et al , 2017Fioriti and Hermanns 2015;Huang et al 2018Huang et al , 2019. Ranking supermartingales are similar to ranking functions, but one requires that the value decreases in expectation.…”