Efficient Timeout Synthesis in Fixed-Delay CTMC Using Policy Iteration

Korenčiak, Ľuboš; Kučera, Antonín; Řehák, Vojtěch

doi:10.1109/mascots.2016.34

Cited by 3 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the prototype implementation, we applied some optimizations mainly computing local upper bounds for each state in the constructed semi-MDP. Also, to achieve better perturbation bounds on the expected mean-payoff, i.e., to compute bounds on expected time and cost to reach some state from all other states, we rely on techniques presented in [5,19]. Using these optimizations, for instance in the experiment of disk drive model, some discretization bounds δ were improved from 2.39 · 10 −239 to 7.03 · 10 −19 .…”

Section: Experimental Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Mean-payoff Optimization in Continuous-time Markov Chains with Parametric Alarms

Baier

Dubslaff

Korenčiak

et al. 2019

ACM Trans. Model. Comput. Simul.

View full text Add to dashboard Cite

Continuous-time Markov chains with alarms (ACTMCs) allow for alarm events that can be non-exponentially distributed. Within parametric ACTMCs, the parameters of alarm-event distributions are not given explicitly and can be subject of parameter synthesis. An algorithm solving the ε-optimal parameter synthesis problem for parametric ACTMCs with long-run average optimization objectives is presented. Our approach is based on reduction of the problem to finding long-run average optimal strategies in semi-Markov decision processes (semi-MDPs) and sufficient discretization of parameter (i.e., action) space. Since the set of actions in the discretized semi-MDP can be very large, a straightforward approach based on explicit action-space construction fails to solve even simple instances of the problem. The presented algorithm uses an enhanced policy iteration on symbolic representations of the action space. The soundness of the algorithm is established for parametric ACTMCs with alarm-event distributions satisfying four mild assumptions that are shown to hold for uniform, Dirac and Weibull distributions in particular, but are satisfied for many other distributions as well. An experimental implementation shows that the symbolic technique substantially improves the efficiency of the synthesis algorithm and allows to solve instances of realistic size.

show abstract

Section: Experimental Evaluationmentioning

confidence: 99%

“…Contrary, the synthesis of appropriate rates in CTMCs was efficiently solved in [15,16,7,9]. A special variant of ACTMC, where only alarms with Dirac distributions are allowed, has been considered in [5,20,19]. Their algorithms synthesize ε-optimal alarm parameters towards an expected reachability objective.…”

Section: Introductionmentioning

confidence: 99%

Mean-payoff Optimization in Continuous-time Markov Chains with Parametric Alarms

Baier

Dubslaff

Korenčiak

et al. 2019

ACM Trans. Model. Comput. Simul.

View full text Add to dashboard Cite

show abstract

CTMCs with Imprecisely Timed Observations

Badings,

Volk,

Junges

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters but may be uncertain. Thus, we consider a setting in which we are given a sequence of imprecisely timed labels called the evidence. The problem is to compute reachability probabilities, which we condition on this evidence. Our key contribution is a method that solves this problem by unfolding the CTMC states over all possible timings for the evidence. We formalize this unfolding as a Markov decision process (MDP) in which each timing for the evidence is reflected by a scheduler. This MDP has infinitely many states and actions in general, making a direct analysis infeasible. Thus, we abstract the continuous MDP into a finite interval MDP (iMDP) and develop an iterative refinement scheme to upper-bound conditional probabilities in the CTMC. We show the feasibility of our method on several numerical benchmarks and discuss key challenges to further enhance the performance.

show abstract

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms

Baier

Dubslaff

Korenčiak

et al. 2017

Quantitative Evaluation of Systems

View full text Add to dashboard Cite

Abstract. Continuous-time Markov chains with alarms (ACTMCs) allow for alarm events that can be non-exponentially distributed. Within parametric ACTMCs, the parameters of alarm-event distributions are not given explicitly and can be subject of parameter synthesis. An algorithm solving the ε-optimal parameter synthesis problem for parametric ACTMCs with long-run average optimization objectives is presented. Our approach is based on reduction of the problem to finding long-run average optimal strategies in semi-Markov decision processes (semi-MDPs) and sufficient discretization of parameter (i.e., action) space. Since the set of actions in the discretized semi-MDP can be very large, a straightforward approach based on explicit action-space construction fails to solve even simple instances of the problem. The presented algorithm uses an enhanced policy iteration on symbolic representations of the action space. The soundness of the algorithm is established for parametric ACTMCs with alarm-event distributions satisfying four mild assumptions that are shown to hold for uniform, Dirac and Weibull distributions in particular, but are satisfied for many other distributions as well. An experimental implementation shows that the symbolic technique substantially improves the efficiency of the synthesis algorithm and allows to solve instances of realistic size.

show abstract

Efficient Timeout Synthesis in Fixed-Delay CTMC Using Policy Iteration

Cited by 3 publications

References 30 publications

Mean-payoff Optimization in Continuous-time Markov Chains with Parametric Alarms

Mean-payoff Optimization in Continuous-time Markov Chains with Parametric Alarms

CTMCs with Imprecisely Timed Observations

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms

Contact Info

Product

Resources

About