Robust Control for Dynamical Systems with Non-Gaussian Noise via Formal Abstractions

Abstract

Controllers for dynamical systems that operate in safety-critical settings must account for stochastic disturbances. Such disturbances are often modeled as process noise in a dynamical system, and common assumptions are that the underlying distributions are known and/or Gaussian. In practice, however, these assumptions may be unrealistic and can lead to poor approximations of the true noise distribution. We present a novel controller synthesis method that does not rely on any explicit representation of the noise distributions. In particular, we address the problem of computing a controller that provides probabilistic guarantees on safely reaching a target, while also avoiding unsafe regions of the state space. First, we abstract the continuous control system into a finite-state model that captures noise by probabilistic transitions between discrete states. As a key contribution, we adapt tools from the scenario approach to compute probably approximately correct (PAC) bounds on these transition probabilities, based on a finite number of samples of the noise. We capture these bounds in the transition probability intervals of a so-called interval Markov decision process (iMDP). This iMDP is, with a user-specified confidence probability, robust against uncertainty in the transition probabilities, and the tightness of the probability intervals can be controlled through the number of samples. We use state-of-the-art verification techniques to provide guarantees on the iMDP and compute a controller for which these guarantees carry over to the original control system. In addition, we develop a tailored computational scheme that reduces the complexity of the synthesis of these guarantees on the iMDP. Benchmarks on realistic control systems show the practical applicability of our method, even when the iMDP has hundreds of millions of transitions.

Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty

“…Model-based approaches. Abstractions of stochastic models are well-studied (Abate et al 2008;Alur et al 2000), with applications to stochastic hybrid Lavaei et al 2022), switched (Lahijanian, Andersson, and Belta 2015), and partially observable systems (Badings et al 2021;Haesaert et al 2018). Various tools exist, e.g., StocHy , ProbReach (Shmarov and Zuliani 2015), and SReachTools (Vinod, Gleason, and Oishi 2019).…”

“…5, we must compute R −1 α (T ℓ ) for each action a ℓ ∈ Act with associated target set T ℓ . In the extended version of this paper (Badings et al 2022b, Appendix A, Lemma 3), we show that R −1 α (T ℓ ) is a polytope characterized by the vertices of U and T ℓ , which is computationally tractable to compute.…”

Correct-by-Construction Control for Stochastic and Uncertain Dynamical Models via Formal Abstractions

“…In [8], we use this method to solve a reach-avoid problem for a UAV operating under turbulence (we compare scenarios with different turbulence levels), represented by stochastic noise of unknown distribution. The UAV is modeled by a 6D dynamical model (we refer to [8] for the explicit model). In Fig.…”

“…By contrast, most other abstraction methods rely on forward reachability computations, which are associated with errors that grow with the time horizon of the property (see related work for details). Our backward scheme avoids such abstraction errors, at the cost of requiring slightly more restrictive assumptions on the model dynamics (see, e.g., [8] for details).…”

Correct-by-Design Control of Parametric Stochastic Systems

“…There is a limited body of work addressing systems with both stochastic and epistemic uncertainties. Badings et al [3] have studied monolithic linear systems with unknown additive noise for reach-avoid specifications. This work is extended in [2] to fully unknown linear systems.…”