Bayes-Adaptive Planning for Data-Efficient Verification of Uncertain Markov Decision Processes

Wijesuriya, Viraj B.; Abate, Alessandro

doi:10.1007/978-3-030-30281-8_6

Cited by 5 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, there are infinitely many constraints in the RCP since x ∈ X, where X is a continuous set. To address the issue of unknown P w and the expectation term in g 4 in (7), we replace the expectation term with its empirical mean approximation by collecting N i.i.d. samples w j , j ∈ {1, .…”

Section: Data-driven Safety Verification Of Stochastic Systemsmentioning

confidence: 99%

See 1 more Smart Citation

Safety Verification of Stochastic Systems: A Repetitive Scenario Approach

Salamati¹

2023

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

In this paper, we develop a data-driven approach for the safety verification of stochastic systems with unknown dynamics. First, we use a notion of barrier certificates in order to cast the safety verification as a robust convex program (RCP). Solving this optimization program is difficult because the model of the stochastic system, which is unknown, appears in one of the constraints. Therefore, we construct a scenario convex program (SCP) by collecting a number of samples from trajectories of the system. Then, we develop a repetition-based scenario framework to provide an out-of-sample performance guarantee for the constructed SCP. In particular, we iteratively solve an SCP for a given number of samples, and then check its feasibility using a certain number of new samples after substituting the optimal decision variables from solving the SCP. We continue the iterations until a desired violation error is achieved. Eventually, a safety condition is checked on top of the feasibility problem. If the safety condition is fulfilled, then we can provide a lower bound on the probability of safety satisfaction for the original stochastic system by leveraging the optimal solution of the successful iteration. We illustrate the effectiveness of the proposed results through a two-tank system case study, where the safety objective is to ensure that the water levels in both tanks are within some safe zones.

show abstract

Section: Data-driven Safety Verification Of Stochastic Systemsmentioning

confidence: 99%

“…Theorem 3: Consider a stochastic system S as in (1), where f and P w are unknown, and a safety specification Ψ as in Definition 2. Assume all constraints in (7) are Lipschitz continuous a with respect to x and with a Lipschitz constant L x . Let ϵ, ϵ ′ ∈ [0, 1], ϵ ′ ≤ ϵ.…”

Section: Safety Verification Of Stochastic Systemsmentioning

confidence: 99%

Safety Verification of Stochastic Systems: A Repetitive Scenario Approach

Salamati¹

2023

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

show abstract

“…Note that the last inequality in (30) encodes the fact that the control input should be inside the set specified by the polytope (28).…”

Section: Data-driven Controller Synthesismentioning

confidence: 99%

“…This approach is extended in [27] by providing a methodology for computing the invariant sets of discrete-time black-box systems. A novel Bayes-adaptive planning algorithm for data-efficient verification of uncertain Markov decision processes is introduced in [28]. A framework is proposed in [29] to provide a formal guarantee on data-driven model identification and controller synthesis.…”

Section: Introductionmentioning

confidence: 99%

Data-driven verification and synthesis of stochastic systems through barrier certificates

Salamati¹,

Lavaei²,

Soudjani³

2021

Preprint

View full text Add to dashboard Cite

In this work, we study verification and synthesis problems for safety specifications over unknown discrete-time stochastic systems. When a model of the system is available, barrier certificates have been successfully applied for ensuring the satisfaction of safety specifications. In this work, we formulate the computation of barrier certificates as a robust convex program (RCP). Solving the acquired RCP is hard in general because the model of the system that appears in one of the constraints of the RCP is unknown. We propose a data-driven approach that replaces the uncountable number of constraints in the RCP with a finite number of constraints by taking finitely many random samples from the trajectories of the system. We thus replace the original RCP with a scenario convex program (SCP) and show how to relate their optimizers. We guarantee that the solution of the SCP is a solution of the RCP with a priori guaranteed confidence when the number of samples is larger than a pre-computed value. This provides a lower bound on the safety probability of the original unknown system together with a controller in the case of synthesis. We also discuss an extension of our verification approach to a case where the associated robust program is non-convex and show how a similar methodology can be applied. Finally, the applicability of our proposed approach is illustrated through three case studies.

show abstract

“…These priors may result from asking different experts which value they would assume for, e.g., the infection rate. The prior may also be the result of Bayesian reasoning [56]. Formally, we capture the uncertainty in the rates by an arbitrary and potentially unknown probability distribution over the parameter space, see Fig.…”

Section: Introductionmentioning

confidence: 99%

Sampling-Based Verification of CTMCs with Uncertain Rates

Badings

Jansen

Junges

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We employ uncertain parametric CTMCs with parametric transition rates and a prior on the parameter values. The prior encodes uncertainty about the actual transition rates, while the parameters allow dependencies between transition rates. Sampling the parameter values from the prior distribution then yields a standard CTMC, for which we may compute relevant reachability probabilities. We provide a principled solution, based on a technique called scenario-optimization, to the following problem: From a finite set of parameter samples and a user-specified confidence level, compute prediction regions on the reachability probabilities. The prediction regions should (with high probability) contain the reachability probabilities of a CTMC induced by any additional sample. To boost the scalability of the approach, we employ standard abstraction techniques and adapt our methodology to support approximate reachability probabilities. Experiments with various well-known benchmarks show the applicability of the approach.

show abstract

Bayes-Adaptive Planning for Data-Efficient Verification of Uncertain Markov Decision Processes

Cited by 5 publications

References 22 publications

Safety Verification of Stochastic Systems: A Repetitive Scenario Approach

Safety Verification of Stochastic Systems: A Repetitive Scenario Approach

Data-driven verification and synthesis of stochastic systems through barrier certificates

Sampling-Based Verification of CTMCs with Uncertain Rates

Contact Info

Product

Resources

About