Data-Efficient Bayesian Verification of Parametric Markov Chains

Abstract

Obtaining complete and accurate models for the formal verification of systems is often hard or impossible. We present a data-based verification approach, for properties expressed in a probabilistic logic, that addresses incomplete model knowledge. We obtain experimental data from a system that can be modelled as a parametric Markov chain. We propose a novel verification algorithm to quantify the confidence the underlying system satisfies a given property of interest by using this data. Given a parameterised model of the system, the procedure first generates a feasible set of parameters corresponding to model instances satisfying a given probabilistic property. Simultaneously, we use Bayesian inference to obtain a probability distribution over the model parameter set from data sampled from the underlying system. The results of both steps are combined to compute a confidence the underlying system satisfies the property. The amount data required is minimised by exploiting partial knowledge of the system. Our approach offers a framework to integrate Bayesian inference and formal verification, and in our experiments our new approach requires one order of magnitude less data than standard statistical model checking to achieve the same confidence.

Automated Experiment Design for Data-Efficient Verification of Parametric Markov Decision Processes

“…This work extends the contributions in [19] by focussing on systems with actions: the presence of (action) non-determinism provides the potential for experiment design, whereby we select actions to improve the accuracy of our confidence value. More precisely we design experiments that maximise the usefulness of the data collected.…”

“…, θ n ) ∈ Θ with θ i ∈ [0, 1], a pMDP is considered linearly parameterised if all outgoing transition probabilities of state-actions pairs have probability g l (θ) or 1 − g l (θ), where g l (θ) = k 0 + k 1 θ 1 + ... + k n θ n with k i ∈ [0, 1] and k i ≤ 1. This restriction is due to the transformations presented in [19] necessary to perform Bayesian inference over the model parameters. As before, ∀s ∈ S, ∀α ∈ Act(s), ∀θ ∈ Θ : s ′ ∈S T θ (s, α, s ′ ) = 1.…”

“…As in [19], we consider linearly parameterised MPDs, where unknown transition probabilities can be linearly related. More precisely, given Θ ⊆ [0, 1] n and parameter vector θ = (θ 1 , .…”

ABC(SMC)$$^2$$: Simultaneous Inference and Model Checking of Chemical Reaction Networks

“…[22] computes the probability that an underlying stochastic system satisfies a given property using data produced by the system and leveraging system's models. Along this line of work, the integration of verification of parameterised discrete-time Markov chains and Bayesian inference is considered in [38], with an extension to Markov decision processes in [39]. Both [38,39] work with small finite-state models with fully observable traces, which allows the posterior probability distribution to be calculated analytically and parameters to be synthesised symbolically.…”

“…Along this line of work, the integration of verification of parameterised discrete-time Markov chains and Bayesian inference is considered in [38], with an extension to Markov decision processes in [39]. Both [38,39] work with small finite-state models with fully observable traces, which allows the posterior probability distribution to be calculated analytically and parameters to be synthesised symbolically. On the contrary, here we work with partially observed data and stochastic models with intractable likelihoods, and must rely on likelihood-free methods and statistical parameter synthesis procedures.…”