Convex Computation of Extremal Invariant Measures of Nonlinear Dynamical Systems and Markov Processes

Abstract

We propose a convex-optimization-based framework for computation of invariant measures of polynomial dynamical systems and Markov processes, in discrete and continuous time. The set of all invariant measures is characterized as the feasible set of an infinite-dimensional linear program (LP). The objective functional of this LP is then used to single-out a specific measure (or a class of measures) extremal with respect to the selected functional such as physical measures, ergodic measures, atomic measures (corresponding to, e.g., periodic orbits) or measures absolutely continuous w.r.t. to a given measure. The infinite-dimensional LP is then approximated using a standard hierarchy of finite-dimensional semidefinite programming problems (SDPs), the solutions of which are truncated moment sequences, which are then used to reconstruct the measure. In particular, we show how to approximate the support of the measure as well as how to construct a sequence of weakly converging absolutely continuous approximations. As a byproduct, we present a simple method to certify the non-existence of an invariant measure, which is an important question in the theory of Markov processes. The presented framework, where a convex functional is minimized or maximized among all invariant measures, can be seen as a generalization of and a computational method to carry out the so called ergodic optimization, where linear functionals are optimized over the set of invariant measures. Finally, we also describe how the presented framework can be adapted to compute eigenmeasures of the Perron-Frobenius operator.

Computing Controlled Invariant Sets from Data Using Convex Optimization

“…Now we formulate the finite-dimensional LP solved by our method, which is lies at the heart of the proposed approach. This LP is nothing but the LP (8) with the constraint imposed only on the available data (2) and with the additional constraint (15) imposed on the artificial data set (16). We write down the LP with an explicit parametrization of the decision variable as v = β(x) c, with c ∈ R N .…”

“…Historically, the idea of using infinite-dimensional linear programming to address nonlinear optimal control problems originated, to the best of our knowledge, with the work of Rubio [23], closely followed by the works of Vinter and Lewis [24,16]. The work of Rubio [23] is in itself a follow-up on his earlier work [22] that use the infinite-dimensional linear-programming embedding to study calculus of variations problems within the framework of generalized curves introduced by Young in [25].…”

“…The first systematic use for optimal control in conjunction with semidefinite programming was in [17]. After that, this approach was successfully applied to a number of problems including region of attraction [10,13], maximum invariant sets [14], invariant measures [16,20], bounding extreme values on attractors [6] or, very recently, nonlinear partial differential equations analysis and control [15,21]. It should be mentioned that the occupation measure approach is also heavily employed in the stochastic processes literature, the survey of which is beyond the scope of this paper; see, e.g., [11,Chapter 6].…”

“…which is the LP (8) with the additional constraint −1 ≤ v ≤ (1 − α) −1 imposed on the artificial data set (16). By assumption 1, the Lipschitz constant of any function v feasible in this problem is bounded by some L < ∞ and the coefficient vector c defining v is bounded by some M .…”

Verification of some functional inequalities via polynomial optimization

“…For example, classical Lyapunov functions that are positive definite and decay along trajectories prove the stability of equilibrium states (Datko, 1970;Lyapunov, 1992). Other types of Lyapunov-like functions, generically called auxiliary functions in this work, can bound the effect of external disturbances (Ahmadi et al, 2016;Dashkovskiy and Mironchenko, 2013;Willems, 1972), certify safety (Ahmadi et al, 2017;Miller et al, 2021;Prajna et al, 2007), approximate reachable sets and basins of attraction (Henrion and Korda, 2014;Korda et al, 2013;Tan and Packard, 2006;Valmorbida and Anderson, 2017), estimate extreme behaviour (Chernyshenko et al, 2014;Fantuzzi and Goluskin, 2020;Fantuzzi et al, 2016;Goluskin, 2020;Goluskin and Fantuzzi, 2019;Korda et al, 2021;Tobasco et al, 2018), and solve optimal control problems (Henrion et al, 2008;Korda et al, 2018;Lasserre et al, 2008).…”