This work deals with Markov processes that are defined over an uncountable state space (possibly hybrid) and embedding nondeterminism in the shape of a control structure. The contribution looks at the problem of optimization, over the set of allowed controls, of probabilistic specifications defined by automatain particular, the focus is on deterministic finite-state automata. This problem can be reformulated as an optimization of a probabilistic reachability property over a product process obtained from the model for the specification and the model of the system. Optimizing over automata-based specifications thus leads to maximal or minimal probabilistic reachability properties. For both setups, the contribution shows that these problems can be sufficiently tackled with history-independent Markov policies. This outcome has relevant computational repercussions: in particular, the work develops a discretization procedure leading into standard optimization problems over Markov decision processes. Such procedure is associated with exact error bounds and is experimentally tested on a case study.

ABSTRACT. This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of ω-regular properties can be reduced to the solution of one of two fundamental problems: reachability and repeated reachability. We provide a comprehensive study of the former problem and an initial characterisation of the (much more involved) latter problem. A case study elucidates concepts and techniques.

No abstract

This work investigates infinite-horizon properties over discrete-time stochastic models with continuous state spaces. The focus is on understanding how the structural features of a model (e.g., the presence of absorbing sets) affect the values of these properties and relate to their uniqueness. Furthermore, we argue that the investigation of these features can lead to approximation bounds for the value of such properties, as well as to improvements on their computation. The article employs the presented results to find a stochastic bisimulation function of two processes.

Results on approximate model-checking of Stochastic Hybrid Systems (SHS) against general temporal specifications lead to abstractions that structurally depend on the given specification or with a state cardinality that crucially depends on the size of the specification. In order to cope with the associated issues of generality and scalability, we propose a specification-free abstraction approach that is general, namely it allows constructing a single abstraction to be then used for a whole cohort of problems. It furthermore computationally outperforms specificationdependent abstractions over linear temporal properties, such as bounded LTL (BLTL). The proposed approach unifies techniques for the approximate abstraction of SHS over different classes of properties by explicitly relating the error introduced by the approximation to the distance between transition kernels of abstract and concrete models, and by propagating the error in time over the horizon of the specification. The new technique is compared over a case study to related results in the literature.