Learning and Designing Stochastic Processes from Logical Constraints

Abstract

Abstract. Stochastic processes offer a flexible mathematical formalism to model and reason about systems. Most analysis tools, however, start from the premises that models are fully specified, so that any parameters controlling the system's dynamics must be known exactly. As this is seldom the case, many methods have been devised over the last decade to infer (learn) such parameters from observations of the state of the system. In this paper, we depart from this approach by assuming that our observations are qualitative properties encoded as satisfaction of linear temporal logic formulae, as opposed to quantitative observations of the state of the system. An important feature of this approach is that it unifies naturally the system identification and the system design problems, where the properties, instead of observations, represent requirements to be satisfied. We develop a principled statistical estimation procedure based on maximising the likelihood of the system's parameters, using recent ideas from statistical machine learning. We demonstrate the efficacy and broad applicability of our method on a range of simple but non-trivial examples, including rumour spreading in social networks and hybrid models of gene regulation.

Machine Learning Methods in Statistical Model Checking and System Design – Tutorial

“…We conclude this section noting that we can consider different classes of stochastic models, including Stochastic Differential Equations and Stochastic Hybrid Systems. We refer the reader to [12,15,14] for a more detailed discussion in this sense.…”

“…In this section, we review a GP-based approach to address these problems, which was first presented in [11,14] for qualitative semantics (problems 1 and 2) and in [13,15] for quantitative semantics (robustness of a formula, problem 3).…”

“…Unfortunately, the log-likelihood (4) is never analytically available, except in the simplest of cases. In [11,14] we proposed an alternative, statistical approach to optimise the uncomputable log-likelihood (4). This is based on obtaining noisy estimates of the function from a limited number of SMC runs, and then adopting a reinforcement learning approach to obtain a provably optimal solution.…”

“…. , ϕ M |θ) for a fixed θ, see [11,14] for details. Robust design, instead, has been modelled in [13,15] as the maximisation of the expected quantitative score for the formula ϕ expressing the desired requirement.…”

“…Recently, we proposed a novel family of algorithms for statistical model checking and parameter synthesis on CTMC models with parametric uncertainty, which can achieve considerable computational savings over parameter exploration methods [11,12,13,14,15]. Our methods are theoretically grounded on a novel characterisation of the functional dependence of the satisfaction probability of a formula on the parameters.…”