No abstract

No abstract

Summary In this paper, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control for nonlinear stochastic dynamical systems. Specifically, we provide a simplified and tutorial framework for stochastic optimal control and focus on connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory. In particular, we show that asymptotic stability in probability of the closed‐loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady‐state form of the stochastic Hamilton–Jacobi–Bellman equation and, hence, guaranteeing both stochastic stability and optimality. In addition, we develop optimal feedback controllers for affine nonlinear systems using an inverse optimality framework tailored to the stochastic stabilization problem. These results are then used to provide extensions of the nonlinear feedback controllers obtained in the literature that minimize general polynomial and multilinear performance criteria. Copyright © 2017 John Wiley & Sons, Ltd.

No abstract

In this paper, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control for nonlinear stochastic dynamical systems. Specifically, we provide a simplified and tutorial framework for stochastic optimal control and focus on connections between stochastic Lyapunov theory and stochastic Hamilton-Jacobi-Bellman theory. In particular, we show that asymptotic stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady-state form of the stochastic Hamilton-Jacobi-Bellman equation and, hence, guaranteeing both stochastic stability and optimality. In addition, we develop optimal feedback controllers for affine nonlinear systems using an inverse optimality framework tailored to the stochastic stabilization problem. These results are then used to provide extensions of the nonlinear feedback controllers obtained in the literature that minimize general polynomial and multilinear performance criteria.OPTIMAL AND INVERSE OPTIMAL STOCHASTIC CONTROL 4725 the Borel -algebra B n . Here, we use the notation x.t/ to represent the stochastic process x.t; !/ omitting its dependence on !.We denote the set of equivalence classes of measurable, integrable, and square-integrable R n or R n m (depending on context) valued random processes on . ; F; P / over the semi-infinite parameter space OE0; 1/ by L 0 . ; F; P /, L 1 . ; F; P /, and L 2 . ; F; P /, respectively, where the equivalence relation is the one induced by P -almost-sure equality. In particular, elements of L 0 . ; F; P / take finite values P -almost surely (a.s.). Hence, depending on the context, R n will denote either the set of n 1 real variables or the subspace of L 0 . ; F; P / comprised of R n random processes that are constant almost surely. All inequalities and equalities involving random processes on . ; F; P / are to be understood to hold P -almost surely. Furthermore, EOE and E x 0 OE denote, respectively, the expectation with respect to the probability measure P and with respect to the classical Wiener measure P x 0 .Finally, we write tr( ) for the trace operator, . / 1 for the inverse operator, V 0 .x/ , @V .x/ @x for the Fréchet derivative of V at x, V 00 .x/ , @ 2 V .x/ @x 2 for the Hessian of V at x, and H n for the Hilbert space of random vectors x 2 R n , that is,For an open set D Â R n , H D n , ¹x 2 H n W x W ! Dº denotes the set of all the random vectors in H n induced by D. Similarly, for every x 0 2 R n , H x 0 n , ¹x 2 H n W x a.s. D x 0 º. Furthermore, C 2 denotes the space of real-valued functions V W D ! R that are two-times continuously differentiable with respect to x 2 D Â R n .Consider the nonlinear stochastic dynamical system G given byfor some Lipschitz constant L > 0, and hence, because x.t 0 / 2 H D n and x.t 0 / are independent of .w.t / w.t 0 //; t > t 0 , it follows that there exists a unique solution x 2 L 2 . ; F; P / of (1) in the Á 0 to (1) is globally asymptotically stable in probability ...