Strategy Representation by Decision Trees in Reactive Synthesis

Abstract

Graph games played by two players over finite-state graphs are central in many problems in computer science. In particular, graph games with ω-regular winning conditions, specified as parity objectives, which can express properties such as safety, liveness, fairness, are the basic framework for verification and synthesis of reactive systems. The decisions for a player at various states of the graph game are represented as strategies. While the algorithmic problem for solving graph games with parity objectives has been widely studied, the most prominent data-structure for strategy representation in graph games has been binary decision diagrams (BDDs). However, due to the bit-level representation, BDDs do not retain the inherent flavor of the decisions of strategies, and are notoriously hard to minimize to obtain succinct representation. In this work we propose decision trees for strategy representation in graph games. Decision trees retain the flavor of decisions of strategies and allow entropy-based minimization to obtain succinct trees. However, decision trees work in settings (e.g., probabilistic models) where errors are allowed, and overfitting of data is typically avoided. In contrast, for strategies in graph games no error is allowed, and the decision tree must represent the entire strategy. We develop new techniques to extend decision trees to overcome the above obstacles, while retaining the entropybased techniques to obtain succinct trees. We have implemented our techniques to extend the existing decision tree solvers. We present experimental results for problems in reactive synthesis to show that decision trees provide a much more efficient data-structure for strategy representation as compared to BDDs.

SOS: Safe, Optimal and Small Strategies for Hybrid Markov Decision Processes

“…Fourthly, DT can use much wider class of predicates, compared to single bit tests for a bit representation in a BDD. This final point is also a reason (together with the smaller size) why DT is a more understandable representation than a BDD [8,9]. We also illustrate this point on a case-study in Remark 1.…”

“…In the context of verification, MDPs are often represented using variants of (MT)BDDs [20,29,40], and strategies by BDDs [56]. Learning a compact decisiontree representation of a strategy has been investigated in [38] for the case of body sensor networks, in [8] for finite (discrete) MDPs, and in [9] for finite games, but only with Boolean variables. Moreover, these decision trees can only predict a single action for a state configuration whereas in this work, we allow the trees to predict more than one action for a single configuration.…”

“…The leaf attribute θ gives, for each action a, the number of configurations in the leaf where the strategy disallows and allows a, respectively. For example, consider a node with 10 configurations with θ = [(0, 10), (2,8), (9,1)]. This means that the first action is allowed by all 10 configurations in the node, the second action is disallowed by 2 configurations and allowed by 8, and the third action is disallowed by 9 configurations and allowed only by 1.…”

“…There are further advantages of DT, related to learning, that make them more compact than BDD in some contexts, e.g. [8,9]. Firstly, they can be learnt fast, using the entropy-based heuristic, compared to the graph processing and variable re-ordering of BDDs.…”

Strategy Representation by Decision Trees with Linear Classifiers

“…As a data structure to represent strategies, there are some desirable properties, which are as follows: (a) succinctness, i.e., small strategies are desirable, since smaller strategies represent efficient controllers; (b) explanatory, i.e., the representation explains the decisions of the strategies. While one standard data structure representation for strategies is binary decision diagrams (BDDs) [2,13], recent works have shown that decision trees [46,40] from machine learning provide an attractive alternative data structure for strategy representation [9,11]. The two key advantages of decision trees are: (a) Decision trees utilize various predicates to make decisions and thus retain the inherent flavor of the decisions of the strategies; and (b) there are entropy-based algorithmic approaches for decision tree minimization [46,40].…”