Mean field approach to stochastic control with partial information

Bensoussan, Alain; Yam, Philip

doi:10.1051/cocv/2021085

Cited by 5 publications

(13 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this subsection, we formulate the conventional POSC [ 11 , 15 ]. The state

and the observation

at time

evolve by the following stochastic differential equations (SDEs):

where

and

obey

and

, respectively,

and

are independent standard Wiener processes, and

is the control.…”

Section: Review Of Partially Observable Stochastic Controlmentioning

confidence: 99%

“…In this subsection, we briefly review the derivation of the optimal control function of the conventional POSC [ 11 , 15 ]. We first define the unnormalized posterior probability density function

.…”

Section: Review Of Partially Observable Stochastic Controlmentioning

confidence: 99%

“…([ 11 , 15 ]) . The optimal control function of POSC is provided by

where

is the Hamiltonian, which is defined by

…”

Section: Review Of Partially Observable Stochastic Controlmentioning

confidence: 99%

“…In this subsection, we propose the mean-field control approach to ML-POSC. We first show that ML-POSC can be converted into a deterministic control of the probability density function, which is similar to the conventional POSC [ 11 , 15 ]. This approach is used in the mean-field control as well [ 13 , 14 , 24 , 25 ].…”

Section: Mean-field Control Approachmentioning

confidence: 99%

“…As a result, unlike the conventional POSC, ML-POSC finds both the optimal state control and the optimal memory dynamics with given memory limitations. Furthermore, we show that the Bellman equation of ML-POSC can be reduced to the Hamilton–Jacobi–Bellman (HJB) equation by employing a trick from the mean-field control theory [ 13 , 14 , 15 ]. While the Bellman equation is a functional differential equation, the HJB equation is a partial differential equation.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Memory-Limited Partially Observable Stochastic Control and Its Mean-Field Control Approach

Tottori

Kobayashi

2022

Entropy

View full text Add to dashboard Cite

Control problems with incomplete information and memory limitation appear in many practical situations. Although partially observable stochastic control (POSC) is a conventional theoretical framework that considers the optimal control problem with incomplete information, it cannot consider memory limitation. Furthermore, POSC cannot be solved in practice except in special cases. In order to address these issues, we propose an alternative theoretical framework, memory-limited POSC (ML-POSC). ML-POSC directly considers memory limitation as well as incomplete information, and it can be solved in practice by employing the technique of mean-field control theory. ML-POSC can generalize the linear-quadratic-Gaussian (LQG) problem to include memory limitation. Because estimation and control are not clearly separated in the LQG problem with memory limitation, the Riccati equation is modified to the partially observable Riccati equation, which improves estimation as well as control. Furthermore, we demonstrate the effectiveness of ML-POSC for a non-LQG problem by comparing it with the local LQG approximation.

show abstract

“…In this subsection, we formulate the conventional POSC [ 11 , 15 ]. The state

and the observation

at time

evolve by the following stochastic differential equations (SDEs):

where

and

obey

and

, respectively,

and

are independent standard Wiener processes, and

is the control.…”

Section: Review Of Partially Observable Stochastic Controlmentioning

confidence: 99%

.…”

Section: Review Of Partially Observable Stochastic Controlmentioning

confidence: 99%

“…([ 11 , 15 ]) . The optimal control function of POSC is provided by

where

is the Hamiltonian, which is defined by

…”

Section: Review Of Partially Observable Stochastic Controlmentioning

confidence: 99%

Section: Mean-field Control Approachmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Memory-Limited Partially Observable Stochastic Control and Its Mean-Field Control Approach

Tottori

Kobayashi

2022

Entropy

View full text Add to dashboard Cite

show abstract

A Probabilistic Method for a Class of Non-Lipschitz BSDEs with Application to Fund Management

Han¹,

Yam²

2022

SIAM J. Control Optim.

View full text Add to dashboard Cite

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Tottori

Kobayashi

2023

Entropy

View full text Add to dashboard Cite

Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. To obtain the optimal control function of ML-POSC, a system of the forward Fokker–Planck (FP) equation and the backward Hamilton–Jacobi–Bellman (HJB) equation needs to be solved. In this work, we first show that the system of HJB-FP equations can be interpreted via Pontryagin’s minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) for ML-POSC. FBSM is one of the most basic algorithms for Pontryagin’s minimum principle, which alternately computes the forward FP equation and the backward HJB equation in ML-POSC. Although the convergence of FBSM is generally not guaranteed in deterministic control and mean-field stochastic control, it is guaranteed in ML-POSC because the coupling of the HJB-FP equations is limited to the optimal control function in ML-POSC.

show abstract

Mean field approach to stochastic control with partial information

Cited by 5 publications

References 15 publications

Memory-Limited Partially Observable Stochastic Control and Its Mean-Field Control Approach

Memory-Limited Partially Observable Stochastic Control and Its Mean-Field Control Approach

A Probabilistic Method for a Class of Non-Lipschitz BSDEs with Application to Fund Management

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Contact Info

Product

Resources

About