Compositional construction of infinite abstractions for networks of stochastic control systems

Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control

2020

Self Cite

We consider the policy synthesis problem for continuous-state controlled Markov processes evolving in discrete time, when the specification is given as a Büchi condition (visit a set of states infinitely often). We decompose computation of the maximal probability of satisfying the Büchi condition into two steps. The first step is to compute the maximal qualitative winning set, from where the Büchi condition can be enforced with probability one. The second step is to find the maximal probability of reaching the already computed qualitative winning set. In contrast with finite-state models, we show that such a computation only gives a lower bound on the maximal probability where the gap can be non-zero. In this paper we focus on approximating the qualitative winning set, while pointing out that the existing approaches for unbounded reachability computation can solve the second step. We provide an abstraction-based technique to approximate the qualitative winning set by simultaneously using an over-and under-approximation of the probabilistic transition relation. Since we are interested in qualitative properties, the abstraction is non-probabilistic; instead, the probabilistic transitions are assumed to be under the control of a (fair) adversary. Thus, we reduce the original policy synthesis problem to a Büchi game under a fairness assumption and characterize upper and lower bounds on winning sets as nested fixed point expressions in the-calculus. This characterization immediately provides a symbolic algorithm scheme. Further, a winning strategy computed on the abstract game can be refined to a policy on the controlled Markov process. We describe a concrete abstraction procedure and demonstrate our algorithm on two case studies. We show that our techniques are able to provide tight approximations to the qualitative winning set for the Van der Pol oscillator and a 3-d Dubins' vehicle. CCS CONCEPTS • Computing methodologies → Computational control theory.

Section: Computation Of the Abstractionmentioning

confidence: 99%

Symbolic controller synthesis for Büchi specifications on stochastic systems

Majumdar

Mallik

Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control

2020

Self Cite

“…Note that for the sake of readability, we assume that Σ i and Σ i both have the same dimension (without performing any model order reductions). But if this is not the case and they have different dimensionality, one can employ the techniques proposed in [LSZ18a] to first reduce the dimension of concrete system, and then apply the proposed results of this paper.…”

Section: Finite-step Stochastic Storage and Simulation Functionsmentioning

confidence: 99%

“…Recently, compositional construction of finite abstractions is presented in [SAM15,LSZ18b] using dynamic Bayesian networks and small-gain type conditions, respectively. Compositional construction of infinite abstractions (reduced-order models) is presented in [LSMZ17,LSZ18a] using small-gain type conditions and dissipativity-type properties of subsystems and their abstractions, respectively, both for discrete-time stochastic control systems. Although [LSMZ17,LSZ18a] deal only with infinite abstractions (reduced-order models), our proposed approach here considers finite Markov decision processes as abstractions which are the main tools for automated synthesis of controllers for complex logical properties.…”

Section: Introductionmentioning

confidence: 99%

Compositional abstraction of large-scale stochastic systems: A relaxed dissipativity approach

Lavaei

Nonlinear Analysis: Hybrid Systems

2020

Self Cite

This paper is concerned with a compositional approach for the construction of finite abstractions (a.k.a. finite Markov decision processes) for networks of discrete-time stochastic control systems that are not necessarily stabilizable. The proposed approach leverages the interconnection topology and finite-step stochastic storage functions, that describe joint dissipativity-type properties of subsystems and their abstractions, in order to establish a finite-step stochastic simulation function between the network and its abstraction. In comparison with the existing notions of simulation functions, a finite-step stochastic simulation function needs to decay only after some finite numbers of steps instead of at each time step. In the first part of the paper, we develop a new type of compositional conditions, which is less conservative than the existing ones, for quantifying the probabilistic error between the interconnection of stochastic control subsystems and that of their abstractions. In particular, using this relaxation via a finite-step stochastic simulation function, it is possible to construct finite abstractions such that stabilizability of each subsystem is not required. In the second part of the paper, we propose an approach to construct finite Markov decision processes (MDPs) together with their corresponding finite-step storage functions for general discrete-time stochastic control systems satisfying an incremental passivablity property. We show that for a particular class of stochastic control systems, the aforementioned property can be readily checked by matrix inequalities. We also construct finite MDPs together with their storage functions for a particular class of nonlinear stochastic control systems. To demonstrate the effectiveness of our proposed results, we first apply our approach to an interconnected system composed of 4 subsystems such that 2 of them are not stabilizable. We then consider a road traffic network in a circular cascade ring composed of 50 cells, each of which has the length of 500 meters with 1 entry and 1 way out, and construct compositionally a finite MDP of the network. We employ the constructed finite abstractions as substitutes to compositionally synthesize policies keeping the density of traffic lower than 20 vehicles per cell. Finally, we apply our proposed technique to a fully connected network of 500 nonlinear subsystems and construct their finite MDPs with guaranteed error bounds on their probabilistic output trajectories.

“…However, the major bottleneck of finite-abstraction techniques is their dependency in the state and input set discretization parameters, and consequently, they suffer from the curse of dimensionality: the computational complexity grows exponentially as the dimension of the system increases. To alleviate this issue, compositional techniques have been introduced in the past few years to construct finite abstractions of interconnected systems based on abstractions of smaller subsystems [HHHK13,SAM17,LSZ20a,LSZ20c,LSZ20d,LSZ19a,LSZ18,LSMZ17,LZ19,LSZ19b,LSZ20b,Lav19,NSZ20a,NZ20].…”

Section: Introductionmentioning

confidence: 99%

Compositional abstraction-based synthesis for continuous-time stochastic hybrid systems

Nejati

European Journal of Control

Zamani

2021

Self Cite

In this paper, we propose a compositional framework for the construction of control barrier functions for networks of continuous-time stochastic hybrid systems enforcing complex logic specifications expressed by finite-state automata. The proposed scheme is based on a notion of so-called pseudo-barrier functions computed for subsystems, by employing which one can synthesize hybrid controllers for interconnected systems enforcing complex specifications over a finite-time horizon. Particularly, we first leverage sufficient small-gain type conditions to compositionally construct control barrier functions for interconnected systems based on the corresponding pseudo-barrier functions computed for subsystems. Then, using the constructed control barrier functions, we provide probabilistic guarantees on the satisfaction of given complex specifications in a bounded time horizon. In this respect, we decompose the given complex specification to simpler reachability tasks based on automata representing the complements of original finite-state automata. We then provide systematic approaches to solve those simpler reachability tasks by computing corresponding pseudo-barrier functions. Two different systematic techniques are provided based on (i) the sum-of-squares (SOS) optimization program and (ii) counterexample guided inductive synthesis (CEGIS) to search for pseudo-barrier functions of subsystems while synthesizing local controllers. We demonstrate the effectiveness of our proposed results by applying them to two large-scale physical case studies including a temperature regulation in a circular network of 1000 rooms and a fully-interconnected Kuramoto network of 100 nonlinear oscillators.