Randomized filtering and Bellman equation in Wasserstein space for partial observation control problem

Abstract

We study a stochastic optimal control problem for a partially observed diffusion. By using the control randomization method in [4], we prove a corresponding randomized dynamic programming principle (DPP) for the value function, which is obtained from a flow property of an associated filter process. This DPP is the key step towards our main result: a characterization of the value function of the partial observation control problem as the unique viscosity solution to the corresponding dynamic programming Hamilton-Jacobi-Bellman (HJB) equation. The latter is formulated as a new, fully non linear partial differential equation on the Wasserstein space of probability measures. An important feature of our approach is that it does not require any non-degeneracy condition on the diffusion coefficient, and no condition is imposed to guarantee existence of a density for the filter process solution to the controlled Zakai equation, as usually done for the separated problem. Finally, we give an explicit solution to our HJB equation in the case of a partially observed non Gaussian linear quadratic model.The coefficients b and σ = (σ V σ W ) are deterministic measurable functions from R n × A into R n and R n×(m+d) , and assumed to satisfy the following standing assumptions.(H1) There exists some positive constant C 1 such that for all x, x ′ ∈ R n , a ∈ A,Under (H1), it is shown by standard arguments that there exists a unique strong solution X t,ξ,α to (1.1), which isF-adapted, and satisfiesfor some positive constant C, whereĒ denotes the expectation with respect toP. The aim is to maximize, over all admissible control processes α ∈ A W , the gain functional:T t f (X t,ξ,α s , α s )ds + g(X t,ξ,α T ) ,where f : R n × A → R, and g : R n → R are continuous functions satisfying the quadratic growth condition:

Stochastic Control with Delayed Information and Related Nonlinear Master Equation

“…Moreover, in Bandini et al [2016b], the authors have derived, in this context of noisy observation, the dynamic programming principle with flow of probability measures as state variable and the verification theorem of their master equation. Since the deterministic control problem studied in Section 2 is a particular case of the noisy observation problem, our master equation and the dynamic programming principle in this section could be seen as a special case of theirs.…”

“…We remark that this case is the intersection of several related works. For example, Hu and Tang [2017] studied the linear quadratic case by using the stochastic maximum principle, see Appendix A; Beneš and Karatzas [1983] derived a similar equation when the measures have a density, see Appendix B; in particular, our master equation (2.12)-(2.13) and the DPP Theorem 2.3 below are already covered by Bandini et al [2016b] as a special case. However, since the arguments here are much simpler due to the special structure, which could be helpful for readers to grasp the main ideas, and more importantly since these arguments will be important for the general case in Section 4, we still provide the details.…”

Backward SDEs for optimal control of partially observed path-dependent stochastic systems: A control randomization approach

“…✷ Remark 2. 3 We mention that no non-degeneracy assumption on the diffusion coefficient σ is imposed, and in particular, some lines or columns of σ may be equal to zero. We can then consider a priori more general model than (2.1) by adding dependence of the coefficients b, σ on another diffusion process M , for example an unobserved and uncontrolled factor (see Application in subsection 2.3.1).…”

“…Note that the introduction of the measure-valued process ρ and its occurrence in the generator and the terminal condition of the BSDE is reminiscent of the separated problem in classical optimal control with partial observation. We study in a companion paper [3] how one can also derive such kind of HJB equation in the context of partially observed Markovian control problems. We note that probabilistic numerical methods have already been designed for BSDEs with constraints similar to (1.7) in [21] and [22].…”

Optimal switching problems with an infinite set of modes: An approach by randomization and constrained backward SDEs

“…an auxiliary or dual problem) and a stochastic target problem related to optimal switching. In the framework of switching problems and associated BSDEs the method was further developed and extended in [18], [17], [19] and later applied to different contexts by many authors, see for instance [36], [33], [2], [1] [21], [12], [11], [20], [22], [3], [4].…”