Value Functions and Transversality Conditions for Infinite Horizon Optimal Control Problems

Abstract

This paper investigates the relationship between the maximum principle with an infinite horizon and dynamic programming and sheds new light upon the role of the transversality condition at infinity as necessary and sufficient conditions for optimality with or without convexity assumptions. We first derive the nonsmooth maximum principle and the adjoint inclusion for the value function as necessary conditions for optimality that exhibit the relationship between the maximum principle and dynamic programming. We then present sufficiency theorems that are consistent with the strengthened maximum principle, employing the adjoint inequalities for the Hamiltonian and the value function. Synthesizing these results, necessary and sufficient conditions for optimality are provided for the convex case. In particular, the role of the transversality conditions at infinity is clarified.

Infinite Horizon Optimal Control of Non-Convex Problems Under State Constraints

“…An alternative approach to the discounted infinite horizon problems consists in modification of the cost function in such a way that restrictions of (x,ū) to intervals [t 0 , T ] are locally optimal for the Bolza problems, cf. [35,40,47]. The presence of the discount factor allows then to pass to the limit of the (finite horizon) maximum principles and to conclude that a costate satisfies the adjoint system, the maximality condition and vanishes at infinity.…”

“…Actually, in [35] V is supposed to be C 1 (which is a too strong request) to get these conclusions, while in [47] it is Lipschitz continuous. Furthermore, for the discounted problems considered in [7,35,40,47] the sensitivity relations helped to write a transversality condition also at the initial time.…”

“…The above setting subsumes the classical infinite horizon optimal control problem when f and U are time independent, L(t, x, u) = e −λt (x, u) for some mapping : R n × R m → R + and λ > 0, t 0 = 0. Infinite horizon problems exhibit many phenomena not arising in the context of finite horizon problems and their study is still going on, even in the absence of state constraints, see [1,2,3,4,5,6,33,36,37,38,40,43,44] and their bibliographies. Among such phenomena let us recall that already in 70ies Halkin, see [32] and also [36], observed that in the necessary optimality conditions for an infinite horizon problem it may happen that the co-state of the maximum principle is different from zero at infinity and that only abnormal maximum principles hold true (even for problems without state constraints).…”

Another view of the maximum principle for infinite-horizon optimal control problems in economics

“…The relationship between the maximum principle with infinite-horizon and the dynamic programming was studied in [49,56]. In the case when the optimal value function…”

A note on the sufficiency of the maximum principle for infinite horizon optimal control problems

“…The existence of solutions was studied by Balder, Lykina, and Pickenhain, for example. The relationship between the maximum principle and dynamic programming was investigated in the work of Sagara and Ye …”