Sequential Convex Programming for the Efficient Verification of Parametric MDPs

Cubuktepe, Murat; Jansen, Nils; Junges, Sebastian; Katoen, Joost-Pieter; Papusha, Ivan; Poonawala, Hasan A.; Topcu, Ufuk

doi:10.1007/978-3-662-54580-5_8

Cited by 32 publications

(31 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the work in parameter synthesis focus on finding one parameter value that satisfies the specification. The approaches involve computing a rational function of the reachability probabilities [11,17,41], utilizing convex optimization [34,40], and sampling-based methods [26,29]. The problem of whether there exists a value in the parameter space that satisfies a reachability specification is ETR-complete 4 [47], and finding a satisfying parameter value is exponential in the number of parameters.…”

Section: Introductionmentioning

confidence: 99%

Scenario-Based Verification of Uncertain MDPs

Cubuktepe

Jansen

Junges

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.

show abstract

Section: Introductionmentioning

confidence: 99%

Scenario-Based Verification of Uncertain MDPs

Cubuktepe

Jansen

Junges

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other works in the area are centred around deciding the validity of boolean formulas depending on the parameter range using SMT solvers or extending these techniques to models that involve nondeterminism [7,14,5,27]. [24] have shown how a six-sided dice can be simulated by repeatedly tossing a coin.…”

Section: Introductionmentioning

confidence: 99%

Accelerated Model Checking of Parametric Markov Chains

Gainer

Hahn

Schewe

2018

Automated Technology for Verification and Analysis

View full text Add to dashboard Cite

Parametric Markov chains occur quite naturally in various applications: they can be used for a conservative analysis of probabilistic systems (no matter how the parameter is chosen, the system works to specification); they can be used to find optimal settings for a parameter; they can be used to visualise the influence of system parameters; and they can be used to make it easy to adjust the analysis for the case that parameters change. Unfortunately, these advancements come at a cost: parametric model checking is-or rather was-often slow. To make the analysis of parametric Markov models scale, we need three ingredients: clever algorithms, the right data structure, and good engineering. Clever algorithms are often the main (or sole) selling point; and we face the trouble that this paper focuses on -the latter ingredients to efficient model checking. Consequently, our easiest claim to fame is in the speedup we have often realised when comparing to the state of the art.

show abstract

“…In the past years, there has been a great effort to solve reachability analysis in pMDPs using symbolic approaches [15,21,37,16,1,36]. These methods generally partition the parameter space in regions, associating each region to the optimal memoryless scheduler that maximizes/minimizes the probability to reach the target state.…”

Section: Introductionmentioning

confidence: 99%

“…This symbolic approach to parameter synthesis has been recently extended to handle also the nondeterministic choice in parametric Markov decision processes (pMDPs) [15,21,37,16,1,36], where each different sequence of inputs can induce a distinct Markov chain, resulting potentially in several multivariate rational functions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parameter-Independent Strategies for pMDPs via POMDPs

Arming

Bartocci

Chatterjee

et al. 2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition probabilities to account for stochastic uncertainties of the environment such as noise or input disturbances. We study pMDPs with reachability objectives where the parameter values are unknown and impossible to measure directly during execution, but there is a probability distribution known over the parameter values. We study for the first time computing parameter-independent strategies that are expectation optimal, i.e., optimize the expected reachability probability under the probability distribution over the parameters. We present an encoding of our problem to partially observable MDPs (POMDPs), i.e., a reduction of our problem to computing optimal strategies in POMDPs. We evaluate our method experimentally on several benchmarks: a motivating (repeated) learner model; a series of benchmarks of varying configurations of a robot moving on a grid; and a consensus protocol.

show abstract

Sequential Convex Programming for the Efficient Verification of Parametric MDPs

Cited by 32 publications

References 35 publications

Scenario-Based Verification of Uncertain MDPs

Scenario-Based Verification of Uncertain MDPs

Accelerated Model Checking of Parametric Markov Chains

Parameter-Independent Strategies for pMDPs via POMDPs

Contact Info

Product

Resources

About