On solving optimal control problems with free initial condition using iterative dynamic programming

Abstract

ptimal control problems encountered in chemical engineering frequently involve situations where the initial values of some of 0 the state variables are not specified or the best initial condition is not known. For example, in optimal control of a fed-batch reactor process, the composition of two reactants at the beginning of the operation may be free to be determined, or the optimal initial volume of solution inside the reactor may not be known. In these situations, apart from establishing the optimal policy for the feed rate, we also want to find the initial condition for the state vector that yields the maximum yield of the desired product.One way of finding the best starting condition is to solve the optimal control problem for different values of initial states. However, a more efficient way is to carry out the optimization simultaneously. Teo et al.(1 991) showed that sequential quadratic programming (SQP) could be used with control parametrization technique to solve for both the optimal control policy and the optimal initial condition. Rehbock et al.(1 999) provided a recent survey of the computational methods based on the control parametrization technique for solving a general class of optimal control problems. Another approach is based on converting differential equations to algebraic equations through discretization of all variables and using SQP to solve the transformed equations (Tanartkit and Kegler, 1995). In this approach, the initial values of state variables are incorporated directly into the main SQP problem. The methods based on solving an approximated problem using SQP are generally very fast, but the global solution may be difficult to achieve when the system under consideration is highly nonlinear or multi-modal, as for example, the fedbatch reactor system studied by Hartig et al. (1995). Also, in the optimization of a bifunctional catalyst reactor problem, Luus et al. (1992) showed that the use of SQP led to over 20 local optima. Therefore, research in the development of alternative methods for solving optimal control problems continues. Recently, Claes et al. (1 999) applied a procedure based on Pontryagin's maximum principle to optimal control of a fed-batch reactor with free initial amount of substrate; Chiou and Wang (1999) used a differential evolution algorithm to establish the optimal solution for a fed-batch polymerization system with some free initial conditions. 'Author to whom correspondence may be addressed: E-mail address: luus@ ecf.utoronto.ca For solving optimal control problems where the initial conditions of some of the state variables are not specified, a procedure based on iterative dynamic programming (IDP) is presented. In this procedure, the free initial conditions are taken to be additional control variables for the first time stage only; then the search for the optimal initial conditions and also the optimal control policy is carried out simultaneously using IDP. The procedure is straightforward, and as illustrated with two nonlinear optimal control problems, for e...

A novel analytical second‐order sensitivity calculation approach using the finite element method for chemical engineering problems

“…where μ is the barrier parameter; and s i , i 2 ν is the i − th element of slack variables s corresponding to inequality constraints C I (Á). As μ tends to zero, the solution of Equation (19) converges to the solution of Equation (17). Applying the Newton method to Karush-Kuhn-Tucker (KKT) conditions derived from Equation (19) gives the following [40] : ð20Þ…”

“…IDP was proposed by Luus in 1989. This method introduced grid discretization and region reduction techniques on the basis of dynamic programming, as well as discretized the original problem from time and space . IDP does not need to calculate system gradient information; therefore, it is a global convergence method.…”

Fed-batch optimization of recombinant α-amylase production by Bacillus subtilis using a modified Markov chain Monte Carlo technique

“…Cuthrell and Biegler [7] employed an orthogonal collocation-based sequential quadratic programming to optimize fed-batch culture for penicillin production (containing four state variables) that was first studied by Lim et al [8]. Mekarapiruk and Luus [9] applied iterative dynamic programming with unspecified initial conditions to optimize feed-rate policy for producing penicillin. Their study evaluated the feed-rate profile together with the initial volume and initial substrate concentration.…”

“…Three deterministic candidates given for the values of the decision variables with a fixed final time at 132 hours and equally step size were obtained from their previous work [10]. Markov Chain Monte Carlo (MCMC) procedures, the Gibbs parameter sampling and the Metropolis-Hasting algorithm, have been recently used to estimate model parameters and decision variables in 14th-order equations for α-amylase and protease producing Bacillus subtilis in fedbatch culture [11] and to estimate a set of decision variables for penicillin fed-batch optimization problem by using a set of initial values given by Mekarapiruk and Luus [9,12].…”

“…Ile Amy : mass fraction of isoleucine in α-amylase {0.053} Ile Bio : mass fraction of isoleucine in biomass protein {0.05999} Ile Pro : mass fraction of isoleucine in protease {0.050} J : objective function value [U/h] J max : maximum theoretical productivity value [U/h] : non-growth associated α-amylase productivity constant [h −1 ] k 7 : conversion factor from α-amylase concentration to activity [×10 −3 U/g] k 8: rate constant for α-amylase lysis by protease [×10 3 g/U(h)] k9 : growth associated protease productivity constant [g Pro/ g cell] k 10 : non-growth associated protease productivity constant [h −1 ] k 11 : conversion of protease concentration to activity [×10 −3 U/g] k 12 : non-growth associated protease productivity constant dur-…”