Enhancement of the Global Convergence of Using Iterative Dynamic Programming To Solve Optimal Control Problems

Abstract

Although iterative dynamic programming (IDP) has been well recognized as a powerful method for finding the true optimal solution to a nonlinear dynamic optimization problem, the global optimality of the IDP solution is still not guaranteed completely. This paper explores enhancing the global convergence of using iterative dynamic programming to solve optimal control problems. Two approaches are employed to enhance the possibility of obtaining the true optimum while reducing the required computation efforts. One approach employs Sobol's quasi-random sequence generator to generate allowable controls and the other utilizes multipass computation. Numerical examples show that the use of multipass IDP computation with small numbers of state grid points and allowable control values can indeed enhance the possibility of obtaining a true optimum. This is particularly true when the allowable control values are generated by using Sobol's quasi-random sequence generator.

A Conceptual Framework for the Process System Synthesis and Design Congruent with Corporate Strategy

“…Owing to the tremendous development of computers in the past decades and the limited applicability of dynamic programming for no-simple-stage processing, the application to the process engineering area has not been given any attention. Recently, the applications have been limited and mainly concerned with solving dynamic or control problems by using iterative dynamic programming and the extension to multiple objective cases are almost never investigated. − The principle itself, however, is still useful in dealing with various sequential process systems. As described later, the value chain concept can be useful if it is combined with “the principle of optimality” focusing on the life-cycle activities.…”

Comments on “Enhancement of the Global Convergence of Using Iterative Dynamic Programming To Solve Optimal Control Problems”

“…Sir: In a recent paper Lin and Hwang (1998) suggested ways of improving the convergence rate of iterative dynamic programming (IDP) and used two optimal control problems for illustration. There are, however, some gaps in their evaluation of existing approaches, so the aim of these comments is to provide some further information to enable the suggestions of the authors to be put into proper perspective.…”

“…However, there are some problems, such as the fed-batch reactor problem as considered by Hartig et al (1995), which require the use of a relatively large number of grid points at each time step to establish the global optimal control policy. The two problems considered by Lin and Hwang (1998) do not fall into the latter category, so there is no need to consider more than a single grid point at each time step. This reduces the number of function evaluations substantially without sacrificing the accuracy of the resulting solution.…”

“…The optimal control problem under consideration is a linear system with a quadratic performance index as given by Nagurka et al (1991), where the dimensionality of the system is chosen to be n ) 8; i.e., there are eight linear differential equations and eight control variables to be determined to minimize a quadratic performance index. A piecewise linear continuous optimal control policy for this system with n ) 100, 200, and 250 has been established successfully with IDP (Luus, 1996), so the case with n ) 8 should be straightforward, and actually it is, contrary to the findings of Lin and Hwang (1998). The authors carried out numerous runs, varying the number of grid points, the number of allowable values for control, and the random number generators, and reported that, when used in a single pass, IDP leads to convergence to within 0.01% of the optimum only if Sobol's quasi-random sequence is used.…”