Optimal control of nonlinear Markov jump systems by control parametrisation technique

Abstract

This paper considers an optimal control problem of nonlinear Markov jump systems with continuous state inequality constraints. Due to the presence of continuous-time Markov chain, no existing computation method is available to solve such an optimal control problem. In this paper, a derandomisation technique is introduced to transform the nonlinear Markov jump system into a deterministic system, which simultaneously gives rise to an equivalent deterministic dynamic optimisation problem. The control parametrisation technique is then used to partition the time horizon into a sequence of subintervals such that the control function is approximated by a piecewise constant function consistent with the partition. The heights of the piecewise constant function on the corresponding subintervals are taken as decision variables to be optimised. In this way, the approximate dynamic optimisation problem is an optimal parameter selection problem, which can be viewed as a finite dimensional optimisation problem. To solve it using a gradient-based optimisation method, the gradient formulas of the cost function and the constraint functions are derived. Finally, a real-world practical problem involving a bioreactor tank model is solved using the method proposed.

Mean Escape Time of Switched Riccati Differential Equations

“…It can be easily confirmed that a switched linear system with Poisson jumps is a Markov jump linear system, which is widely studied in the literature (see, e.g., [13,14,15,16]). Although we can find extensive amount of works for studying the Markov jump nonlinear systems [17,18,19,20], most of them implicitly assume the non-existence of the finite-time escape in each of the subsystems. Therefore, these results available in the literature for Markov jump nonlinear systems are not directly applicable to switched RDEs studied in this paper.…”

“…Example 4.7. We first apply Algorithm 1 to the scalar RDE (18). The algorithm gives the graph of the escape time as a function of initial states as in Fig.…”