Stochastic Optimal Control of Delay Equations Arising in Advertising Models

Abstract

We consider a class of optimal control problems of stochastic delay differential equations (SDDE) that arise in connection with optimal advertising under uncertainty for the introduction of a new product to the market, generalizing classical work of Nerlove and Arrow [30]. In particular, we deal with controlled SDDE where the delay enters both the state and the control. Following ideas of Vinter and Kwong [34] (which however hold only in the deterministic case), we reformulate the problem as an infinite dimensional stochastic control problem to which we associate, through the dynamic programming principle, a second order Hamilton-Jacobi-Bellman equation. We show a verification theorem and we exhibit some simple cases where such equation admits an explicit smooth solution, allowing us to construct optimal feedback controls.

On Controlled Linear Diffusions with Delay in a Model of Optimal Advertising under Uncertainty with Memory Effects

“…Problems with memory effects in both the state and the control have been studied first by Vinter and Kwong [35] (in a deterministic LQ setting), and by Gozzi and Marinelli [16] (in the case of linear stochastic dynamics and general objective function). A general theory of solvability of corresponding HJB equations is currently not available, while an infinitedimensional Markovian reformulation and a "smooth" verification theorem have been proved in [16].…”

“…A general theory of solvability of corresponding HJB equations is currently not available, while an infinitedimensional Markovian reformulation and a "smooth" verification theorem have been proved in [16].…”

“…This paper is devoted to the study of a class of optimal control problems for linear stochastic differential equations with delay both in the state and the control term, and is a natural continuation of [16]. These problems arise in the theory of optimal advertising under uncertainty with memory structures.…”

“…The results of [16] are the following: we construct a controlled infinite dimensional SDE that is equivalent to the controlled SDE with delay, we prove a verification theorem, and we exhibit a simple example for which the Bellman equation associated to the control problem admits a sufficiently regular solution, hence the verification theorem can be applied. In the present manuscript, we extend [16] by developing several new sets of results. On the one hand, we provide qualitative characterization of the first-and second-order properties of the optimal value function (Section 3.1).…”

“…More precisely, we formulate an optimization program that seeks to maximize the goodwill level at a given time T > 0 net of the cumulative cost of advertising until T . This optimization problem is studied using techniques of stochastic optimal control in infinite dimensions, using the modeling approach of [16]: in particular, we specify the goodwill dynamics in terms of a controlled stochastic delay differential equation (SDDE), that can be rewritten as a stochastic differential equation (without delay) in a suitable Hilbert space. This allows us to associate to the original control problem for the SDDE an equivalent infinite dimensional control problem for the "lifted" stochastic equation.…”