Nonlinear–nonquadratic optimal and inverse optimal control for stochastic dynamical systems

Abstract

Summary In this paper, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control for nonlinear stochastic dynamical systems. Specifically, we provide a simplified and tutorial framework for stochastic optimal control and focus on connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory. In particular, we show that asymptotic stability in probability of the closed‐loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady‐state form of the stochastic Hamilton–Jacobi–Bellman equation and, hence, guaranteeing both stochastic stability and optimality. In addition, we develop optimal feedback controllers for affine nonlinear systems using an inverse optimality framework tailored to the stochastic stabilization problem. These results are then used to provide extensions of the nonlinear feedback controllers obtained in the literature that minimize general polynomial and multilinear performance criteria. Copyright © 2017 John Wiley & Sons, Ltd.

Control Optimization of Stochastic Systems Based on Adaptive Correction CKF Algorithm

“…Nonlinear Time-Varying Discrete Stochastic Systems. The stochastic control system has the characteristics of uncertainty [30], such as the autonomous control system of unmanned aerial vehicle (UAV) [31], which is affected by unstable wind [32], electromagnetic interference [33], noise signal [34], and so on in the process of operation, resulting in its control system with nonlinear, discrete, time-varying, stochastic, and other characteristics [35]. A stochastic system can be represented in the form of Equations (1) and (2) [36].…”

Implications of dissipativity, inverse optimal control, and stability margins for nonlinear stochastic feedback regulators

“…In a recent paper by Rajpurohit and Haddad, the authors presented a framework for analyzing and designing feedback controllers for nonlinear stochastic dynamical systems. Specifically, a stochastic feedback control problem over an infinite horizon involving a nonlinear‐nonquadratic performance functional was considered and the performance functional was evaluated in closed form as long as the nonlinear‐nonquadratic cost functional considered was related in a specific way to an underlying Lyapunov function that guarantees asymptotic stability in probability of the nonlinear closed‐loop system.…”

“…The approach in the work of the aforementioned authors focused on the role of the Lyapunov function guaranteeing stochastic stability of the closed‐loop system and its connection to the steady‐state solution of the stochastic Hamilton‐Jacobi‐Bellman equation characterizing the optimal nonlinear feedback controller. In order to avoid the complexity in solving the stochastic steady‐state, Hamilton‐Jacobi‐Bellman equation, we do not attempt to minimize a given given cost functional, but rather, we parameterize a family of stochastically stabilizing controllers that minimizes a derived cost functional that provides the flexibility in specifying the control law.…”

“…In order to avoid the complexity in solving the stochastic steady‐state, Hamilton‐Jacobi‐Bellman equation, we do not attempt to minimize a given given cost functional, but rather, we parameterize a family of stochastically stabilizing controllers that minimizes a derived cost functional that provides the flexibility in specifying the control law. This corresponds to addressing an inverse optimal stochastic control problem …”

“…Using the framework developed in Rajpurohit and Haddad, in this paper, we derive stability margins for optimal and inverse optimal nonlinear stochastic feedback regulators. Specifically, sufficient conditions for gain, sector, and disk margin guarantees are obtained for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton‐Jacobi‐Bellman controllers that minimize a nonlinear‐nonquadratic performance criterion with cross‐weighting terms.…”