Discrete LQR and ILQR methods based on high order Runge-Kutta methods

Abstract

In this paper, discrete linear quadratic regulator (DLQR) and iterative linear quadratic regulator (ILQR) methods based on high-order Runge-Kutta (RK) discretization are proposed for solving linear and nonlinear quadratic optimal control problems respectively. As discovered in [W. Hager, Runge-Kutta method in optimal control and the discrete adjoint system, Numer. Math., 2000, pp. 247-282], direct approach with RK discretization is equivalent with indirect approach based on symplectic partitioned Runge-Kutta (SPRK) integration. In this paper, we will reconstruct this equivalence by the analogue of continuous and discrete dynamic programming. Then, based on the equivalence, we discuss the issue that the internal-stage controls produced by direct approach may have lower order accuracy than the RK method used. We propose order conditions for internal-stage controls and then demonstrate that third or fourth order explicit RK discretization cannot avoid the order reduction phenomenon. To overcome this obstacle, we calculate node control instead of internal-stage controls in DLQR and ILQR methods. And numerical examples will illustrate the validity of our methods. Another advantage of our methods is high computational efficiency which comes from the usage of feedback technique. In this paper, we also demonstrate that ILQR is essentially a quasi-Newton method with linear convergence rate.