Nonlinear–nonquadratic optimal and inverse optimal control for discrete‐time stochastic dynamical systems

Abstract

In this article, we investigate the role of Lyapunov functions in evaluating nonlinear-nonquadratic cost functionals for Itô-type nonlinear stochastic difference equations. Specifically, it is shown that the cost functional can be evaluated in closed-form as long as the cost functional is related in a specific way to an underlying Lyapunov function that guarantees asymptotic stability in probability. This result is then used to analyze discrete-time linear as well as nonlinear stochastic dynamical systems with polynomial and multilinear cost functionals. Furthermore, a stochastic optimal control framework is developed by exploiting connections between stochastic Lyapunov theory and stochastic Bellman theory. In particular, we show that asymptotic and geometric stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady state form of the stochastic Bellman equation, and hence, guaranteeing both stochastic stability and optimality.

A Stochastic Discrete Fractional Cournot Duopoly Game: Modeling, Stability, and Optimal Control

“…Theorem 7 (see [48]). Consider the following controlled stochastic discrete dynamical system of the system (3) is given by…”

“…Te resulting sequence F k 􏼈 􏼉 k∈N + of σ-algebras is a fltration on the probability space (Ω, F, P). Moreover, for every k ∈ N + , w(k) is independent of the σ-algebra F l for all 0 ≤ l ≤ k [48].…”

“…Remark 8. For further details on optimal control, including the construction of the Hamiltonian function, see [48].…”

Nonlinear Optimal Control for Stochastic Dynamical Systems

“…If, in addition, (18) holds for all x 0 ∈ R n , then the equilibrium solution x(•) ≡ x e to (11) is globally exponentially p-stable in probability. Finally, if p = 2, we say that the equilibrium solution x(•) ≡ x e to (11) is globally exponentially mean square stable in probability.…”

“…Expanding on the findings of [15,16,18,19], this paper introduces a framework for analyzing and designing feedback controllers for nonlinear stochastic dynamical systems. Specifically, it addresses a feedback stochastic optimal control problem with a nonlinearnonquadratic performance measure over an infinite horizon.…”