Goal Probability Analysis in Probabilistic Planning: Exploring and Enhancing the State of the Art

Steinmetz, Marcel; Hoffmann, Joerg; Buffet, Olivier

doi:10.1613/jair.5153

Cited by 30 publications

(51 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Planning algorithms aim to find strategies which perform well (or even optimally) for a given objective. These algorithms typically assume that a goal is reached eventually [41,45]. This however is unrealistic in many scenarios, e.g.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-cost Bounded Reachability in MDP

Hartmanns

Junges

Katoen

et al. 2018

Tools and Algorithms for the Construction and Analysis of Systems

View full text Add to dashboard Cite

Abstract. We provide an efficient algorithm for multi-objective modelchecking problems on Markov decision processes (MDPs) with multiple cost structures. The key problem at hand is to check whether there exists a scheduler for a given MDP such that all objectives over cost vectors are fulfilled. Reachability and expected cost objectives are covered and can be mixed. Empirical evaluation shows the algorithm's scalability. We discuss the need for output beyond Pareto curves and exploit the available information from the algorithm to support decision makers.

show abstract

Section: Introductionmentioning

confidence: 99%

“…In [27], three different model checking approaches are explored and compared. A survey for heuristic approaches is given in [45]. A Q-learning based approach is described in [13].…”

Section: Introductionmentioning

confidence: 99%

Multi-cost Bounded Reachability in MDP

Hartmanns

Junges

Katoen

et al. 2018

Tools and Algorithms for the Construction and Analysis of Systems

View full text Add to dashboard Cite

show abstract

“…Symbolic implementations of the iterative algorithms should also be tested in practice. In a more theoretical direction, we observe that the planning community has also studied maximizing the probability of reaching a target set of states under the name of MAXPROB (see, e.g., [17,22]). There, online approximations of the NWR would make more sense than the under-approximation we have proposed here.…”

Section: Discussionmentioning

confidence: 99%

The Complexity of Graph-Based Reductions for Reachability in Markov Decision Processes

Roux

Pérez

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We study the never-worse relation (NWR) for Markov decision processes with an infinite-horizon reachability objective. A state q is never worse than a state p if the maximal probability of reaching the target set of states from p is at most the same value from q, regardless of the probabilities labelling the transitions. Extremal-probability states, end components, and essential states are all special cases of the equivalence relation induced by the NWR. Using the NWR, states in the same equivalence class can be collapsed. Then, actions leading to sub-optimal states can be removed. We show that the natural decision problem associated to computing the NWR is coNP-complete. Finally, we extend a previously known incomplete polynomial-time iterative algorithm to underapproximate the NWR.

show abstract

“…Unfortunately, it is EXPTIME-complete to find such an optimal policy in general [28]. Furthermore, recent experiments have shown that, even with very specific restrictions on the action model, finding an optimal policy for a penetration testing task is feasible only for small networks of up to 25 hosts [43]. For the sake of scalability and following the lines of Stackelberg Planning [42], we thus focus on finding critical attack paths, instead of entire policies.…”

Section: Mitigation Analysis As Stackelberg Planningmentioning

confidence: 99%

Towards automated network mitigation analysis

Speicher

Steinmetz

Hoffmann

et al. 2019

Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing

Self Cite

View full text Add to dashboard Cite

Penetration testing is a well-established practical concept for the identification of potentially exploitable security weaknesses and an important component of a security audit. Providing a holistic security assessment for networks consisting of several hundreds hosts is hardly feasible though without some sort of mechanization. Mitigation, prioritizing counter-measures subject to a given budget, currently lacks a solid theoretical understanding and is hence more art than science. In this work, we propose the first approach for conducting comprehensive what-if analyses in order to reason about mitigation in a conceptually well-founded manner. To evaluate and compare mitigation strategies, we use simulated penetration testing, i.e., automated attack-finding, based on a network model to which a subset of a given set of mitigation actions, e.g., changes to the network topology, system updates, configuration changes etc. is applied. Using Stackelberg planning, we determine optimal combinations that minimize the maximal attacker success (similar to a Stackelberg game), and thus provide a well-founded basis for a holistic mitigation strategy. We show that these Stackelberg planning models can largely be derived from network scan, public vulnerability databases and manual inspection with various degrees of automation and detail, and we simulate mitigation analysis on networks of different size and vulnerability.The first version of this article was published on arXiv under the title 'Simulated Penetration Testing and Mitigation Analysis' [2]. The mitigation analysis formalism was later dubbed 'Stackelberg planning' and discussed in a more general scope in a separate publication [42]. The present version thus concentrates on the application to simulated pentesting. In comparison to the previous version, the algorithmic implementation was removed (it can be found [42]), the presentation was streamlined, typos were fixed and the title changed to reflect the new focus. Speicher et al.to obtain a good middle ground between accuracy and practicality [12,17] (we discuss this in detail as part of our related work discussion, Section 2).Simulated pentesting has been used to great success, but an important feature was overseen so far. If a model of the network is given, one can reason about possible mitigations without implementing them -namely, by simulating the attacker on a modified model. This allows for analysing and comparing different mitigation strategies in terms of the (hypothetical) network resulting from their application. This problem was recently introduced as Stackelberg planning in the AI community [42]. Algorithmically, the attacker-planning problem now becomes part of a larger what-if planning problem, in which the best mitigation plans are constructed. This min-max notion is similar to a Stackelberg game, which are frequently used in security games [26]. The foundational assumption is that the defender acts first, while the adversary can choose her best response after observing this choice, similar to a market ...

show abstract

Goal Probability Analysis in Probabilistic Planning: Exploring and Enhancing the State of the Art

Cited by 30 publications

References 50 publications

Multi-cost Bounded Reachability in MDP

Multi-cost Bounded Reachability in MDP

The Complexity of Graph-Based Reductions for Reachability in Markov Decision Processes

Towards automated network mitigation analysis

Contact Info

Product

Resources

About