On time-inconsistent stochastic control in continuous time

Björk, Tomas; Khapko, Mariana; Murgoci, Agatha

doi:10.1007/s00780-017-0327-5

Cited by 295 publications

(390 citation statements)

References 25 publications

Supporting

Mentioning

384

Contrasting

Order By: Relevance

“…This means that P t,µ T converges to µ in P 2 (R d ) when t ր T , uniformly in α ∈ A. Now, from the definition of v in (3.8), we have 8) from the growth condition on f in (H2'). By the continuity assumption on g together with the growth condition on g in (H2'), which allows to use dominated convergence theorem, we deduce from (5.7) thatĝ(P t,µ T ) converges toĝ(µ) when t ր T , uniformly in α ∈ A.…”

Section: Of Test Functions For the Lifted Bellman Equation As The Sementioning

confidence: 99%

“…It is claimed in [8] and [38] that Bellman optimality principle does not hold, and therefore the problem is time-inconsistent. This is correct when one takes into account only the state process X (that is its realization), since it is not Markovian, but as shown in this section, dynamic programming principle holds true whenever we consider the marginal distribution as state variable.…”

Section: Dynamic Programming Principlementioning

confidence: 99%

See 1 more Smart Citation

Bellman equation and viscosity solutions for mean-field stochastic control problem

2018

View full text Add to dashboard Cite

We consider the stochastic optimal control problem of McKean-Vlasov stochastic differential equation where the coefficients may depend upon the joint law of the state and control. By using feedback controls, we reformulate the problem into a deterministic control problem with only the marginal distribution of the process as controlled state variable, and prove that dynamic programming principle holds in its general form. Then, by relying on the notion of differentiability with respect to probability measures recently introduced by P.L. Lions in [32], and a special Itô formula for flows of probability measures, we derive the (dynamic programming) Bellman equation for mean-field stochastic control problem, and prove a verification theorem in our McKeanVlasov framework. We give explicit solutions to the Bellman equation for the linear quadratic mean-field control problem, with applications to the mean-variance portfolio selection and a systemic risk model. We also consider a notion of lifted viscosity solutions for the Bellman equation, and show the viscosity property and uniqueness of the value function to the McKean-Vlasov control problem. Finally, we consider the case of McKean-Vlasov control problem with open-loop controls and discuss the associated dynamic programming equation that we compare with the case of closed-loop controls.MSC Classification: 93E20, 60H30, 60K35.

show abstract

Section: Of Test Functions For the Lifted Bellman Equation As The Sementioning

confidence: 99%

Section: Dynamic Programming Principlementioning

confidence: 99%

Bellman equation and viscosity solutions for mean-field stochastic control problem

2018

View full text Add to dashboard Cite

show abstract

“…The last definition corresponds to the closed-loop or statefeedback type control. This can be characterized by solving the HJB equation obtained by discretizing the corresponding optimal control problem, which is closely related to the multiperson differential game [2], [3], [8], [9], [14]- [18], [24], [35], [36]. In addition, the mixed equilibrium solution concept was used in [37], and the Markovian framework for timeinconsistent linear-quadratic problems was developed in [38].…”

Section: Introductionmentioning

confidence: 99%

Linear-Quadratic Time-Inconsistent Mean-Field Type Stackelberg Differential Games: Time-Consistent Open-Loop Solutions

Moon

Yang

2021

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

In this paper, we consider the linear-quadratic timeinconsistent mean-field leader-follower Stackelberg stochastic differential game with an adapted open-loop information structure. Given a controlled linear stochastic differential equation (SDE), the quadratic objective functionals of the leader and the follower include conditional expectations of state and control (mean field) variables. In addition, the cost parameters could be general nonexponential discounting depending on the initial time. As stated in the existing literature, these two general settings of the objective functionals induce time inconsistency in the optimal solutions. Given an arbitrary control of the leader, we first obtain the follower's (time-consistent) equilibrium control and its state feedback representation in terms of the nonsymmetric coupled Riccati differential equations (RDEs) and the backward SDE. This provides the rational behavior of the follower, characterized by the forward-backward SDE (FBSDE). We then obtain the leader's explicit (time-consistent) equilibrium control and its state feedback representation in terms of the nonsymmetric coupled RDEs, where the constraint of the leader's problem is the FBSDE induced by the follower's rational behavior. With the solvability of the nonsymmetric coupled RDEs, the equilibrium controls of the leader and the follower constitute the time-consistent Stackelberg equilibrium of the paper. Finally, the numerical examples are provided to check the solvability of the nonsymmetric coupled RDEs of the leader and the follower.Index Terms-time-inconsistent stochastic control problem, equilibrium control, Stackelberg differential games, mean-field stochastic systems. 4 This implies that the magnitude of the stochastic noise is controlled by the leader and the follower, which can be viewed as multiplicative noise of the system [51], [52]. 5 As mentioned, an equilibrium control is time consistent; see Definition 2 or [3, Definition 4.1] and [26, Definition 2.1].

show abstract

“…This type of systems of equations appear in time-inconsistent stochastic control problems and characterize their subgame perfect Nash equilibria. Time-inconsistent control problems are recently studied by Ekeland and Lazrak (2010) [6], Yong (2012) [20], Björk, Murgoci and Zhou (2014) [2], Zhou (2012, 2017) [8,9], Djehiche and Huang (2016) [5], Björk, Khapko and Murgoci (2017) [1], Wei, Yong and Yu (2017) [18], Ni, Zhang and Krstic (2018) [13], among others. Time-inconsistency occurs for example when a non-exponential discount rate is considered or when the cost functional is a nonlinear function of (conditional) expectation of a state process such as dynamic mean-variance control problems.…”

Section: Introductionmentioning

confidence: 99%

Small-Time Solvability of a Flow of Forward–Backward Stochastic Differential Equations

Hamaguchi

2020

Appl Math Optim

View full text Add to dashboard Cite

Motivated from time-inconsistent stochastic control problems, we introduce a new type of coupled forward-backward stochastic systems, namely, flows of forward-backward stochastic differential equations. They are systems consisting of a single forward SDE and a continuum of BSDEs, which are defined on different time-intervals and connected via an equilibrium condition. We formulate a notion of equilibrium solutions in a general framework and prove small-time well-posedness of the equations. We also consider discretized flows and show that their equilibrium solutions approximate the original one, together with an estimate of the convergence rate. satisfies lim inf ǫ↓0 J t (u t,ǫ,v ;x t ) − J t (û;x t ) ǫ ≥ 0 for any t ∈ [0, T ) and control v, where u t,ǫ,v is the "spike variation" ofû at time t with respect to v, namely, u t,ǫ,v s := v s if s ∈ [t, t + ǫ) and u t,ǫ,v s:=û s otherwise. Then, by a version of the stochastic maximum principle (see [3,5]), a subgame perfect Nash equilibrium strategŷ u must satisfy the relationfor any t ∈ [0, T ) and v ∈ U. Here, the function H is the Hamiltonian defined by

show abstract

On time-inconsistent stochastic control in continuous time

Cited by 295 publications

References 25 publications

Bellman equation and viscosity solutions for mean-field stochastic control problem

Bellman equation and viscosity solutions for mean-field stochastic control problem

Linear-Quadratic Time-Inconsistent Mean-Field Type Stackelberg Differential Games: Time-Consistent Open-Loop Solutions

Small-Time Solvability of a Flow of Forward–Backward Stochastic Differential Equations

Contact Info

Product

Resources

About