Dynamic Programming for General Linear Quadratic Optimal Stochastic Control with Random Coefficients

Abstract

Abstract. We are concerned with the linear-quadratic optimal stochastic control problem where all the coefficients of the control system and the running weighting matrices in the cost functional are allowed to be predictable (but essentially bounded) processes and the terminal state-weighting matrix in the cost functional is allowed to be random. Under suitable conditions, we prove that the value field V (t, x, ω), (t, x, ω) ∈ [0, T ] × R n × Ω, is quadratic in x, and has the following form: V (t, x) = Ktx, x where K is an essentially bounded nonnegative symmetric matrix-valued adapted processes. Using the dynamic programming principle (DPP), we prove that K is a continuous semimartingale of the formwith k being a continuous process of bounded variation and (2003), pp. 53-75] via the stochastic maximum principle with a viewpoint of stochastic flow for the associated stochastic Hamiltonian system. The present paper is its companion, and gives the second but more comprehensive (seemingly much simpler, but appealing to the advanced tool of Doob-Meyer decomposition theorem, in addition to the DDP) adapted solution to a general BSRE via the DDP. Further extensions to the jump-diffusion control system and to the general nonlinear control system are possible.Key words. linear quadratic optimal stochastic control, random coefficients, Riccati equation, backward stochastic differential equations, dynamic programming, semi-martingale AMS subject classifications. 93E20, 49K45, 49N10, 60H101. Formulation of the problem and basic assumptions. Consider the following linear quadratic optimal stochastic control (SLQ in short form) problem: minimize over u ∈ L

Control of Linear Stochastic Systems with State Dependent Noise∗∗Research supported in part by NSF grants DMS 1108884 and DMS 1411412, AFOSR grant FA9550-12-1-0384, ARO grant W911NF-1410390.

“…For simplification of the presentation it is assumed that the stochastic Riccati equation (8) has an adapted solution. While this existence and uniqueness of solutions has been verified using results from backward stochastic differential equations Tang [2014], it can also be obtained from the existence and the uniqueness of the DoobMeyer decomposition of submartingales for the control problem.…”

“…Since there is uniqueness of an optimal control from the quadratic property of the cost and the solution of the Riccati equation enters linearly in the optimal control it follows that there is only one solution of the Riccati equation. Taking expectation of both sides of (11) and realizing that this equality is a translation by the last term in (11) of the Doob-Meyer decomposition of the submartingale of the cost for an arbitrary admissible control and the martingale of the cost for an optimal control, it follows that an optimal admissible control U * is U * (t) = −R −1 (t)B T (t)P (t)X(t) MICNON 2015June 24-26, 2015 …”

“…An explanation of this occurrence with generalizations is given in the 2013 Reid Lecture and this approach is used in to determine optimal controls for linear systems driven by arbitrary square integrable noise processes. Tang Tang [2003] Tang [2014 has used in the latter paper the theory of backward stochastic differential equations to solve a stochastic Riccati equation for a stochastic linear quadratic control problem with linear state and control dependent noise driving the system. In this paper a direct approach is used to obtain an optimal control that is similar to the method in for linear stochastic systems with additive noise processes.…”

Backward Stochastic Riccati Equation with Jumps Associated with Stochastic Linear Quadratic Optimal Control with Jumps and Random Coefficients

“…Until 2013, Tang [25] generally solved this open problem applying the stochastic maximum principle and using the technique of stochastic flow for the associated stochastic Hamiltonian system. In 2015, Tang [26] gives the second but more comprehensive (seeming much simpler, by Doob-Meyer decomposition theorem and Dynamic programming principle) method to solve the general BSREs. For earlier history on BSRE, we refer to Peng [22], Tang and Kohlmann [12,13], Tang [25] and the plenary lecture reported by Peng [21] at the ICM in 2010.…”

“…Obviously (1.6) is positive once the positive definiteness of N obtained. The inverse flow of the controlled stochastic differential equation on interval [0, T ] is a key technique in Tang's method in [26] to give the representation of the BSREJ. In some literature about stochastic differential with jumps [7,16,27,3], the authors give a technical condition to guarantee its inverse flow exists on [0, T ] (using the notation of SDE (2.3))…”