Learning Locomotion Controllers for Walking Using Deep FBSDE

Dai, Bolun; Surabhi, Virinchi Roy; Krishnamurthy, Prashanth; Khorrami, Farshad

doi:10.48550/arxiv.2107.07931

Cited by 2 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the literature it was first applied experimentally to FBSDE in the master thesis [47] and thereafter in [48], both for inverted pendulums, in [49] for an application to attitude control of hypersonic unmanned aerial vehicles. More examples of deep FBSDE are found in [39,40].…”

Section: A Direct Extension Of the Deep Bsde Methods And Why It Failsmentioning

confidence: 98%

“…Therefore, the focus is on global algorithms operating forward in time, with a structure similar to the Deep BSDE solver. There are several attempts to use the deep BSDE solver for stochastic control problem, see e.g., [47,48,49,39,40], where the deep BSDE solver is applied to the FBSDE associated with the stochastic control problem. We demonstrate in this paper, that this approach is problematic since even though an accurate approximation of the control problem can be achieved, this does not imply an approximation of the FBSDE in general.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convergence of a robust deep FBSDE method for stochastic control

Andersson¹,

Andersson²,

Oosterlee³

2022

Preprint

View full text Add to dashboard Cite

In this paper we propose a deep learning based numerical scheme for strongly coupled FBSDE, stemming from stochastic control. It is a modification of the deep BSDE method in which the initial value to the backward equation is not a free parameter, and with a new loss function being the weighted sum of the cost of the control problem, and a variance term which coincides with the means square error in the terminal condition. We show by a numerical example that a direct extension of the classical deep BSDE method to FBSDE, fails for a simple linear-quadratic control problem, and motivate why the new method works. Under regularity and boundedness assumptions on the exact controls of time continuous and time discrete control problems we provide an error analysis for our method. We show empirically that the method converges for three different problems, one being the one that failed for a direct extension of the deep BSDE method.

show abstract

Section: A Direct Extension Of the Deep Bsde Methods And Why It Failsmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

Convergence of a robust deep FBSDE method for stochastic control

Andersson¹,

Andersson²,

Oosterlee³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Non-symmetric problems in the literature are [103,83,102,101,86], and the dimensions are 4, 5, 3, 4, and 2, respectively. A symmetric problem in 100-dimensions is found in [83].…”

Section: Example With Control In Lower Dimensions Than the Statementioning

confidence: 99%

“…We demonstrate that the approach taken in e.g., [100,101,102,83,103], where the deep BSDE method is applied to the FBSDEs associated with the stochastic control problem, is problematic. As we show in the present chapter, even though an accurate approximation of the control problem can be achieved, this does not imply an approximation of the FBSDE in general.…”

Section: Introductionmentioning

confidence: 99%

Neural networks for stochastic control and decision making in mathematical finance

Andersson

View full text Add to dashboard Cite

As the title suggests, this thesis is devoted to machine learning algorithms for stochastic control with applications in mathematical finance. In the first two chapters, we utilize neural networks to find the optimal exercise strategies of high-dimensional American options. Moreover, we compute option values along stochastic trajectories of the underlying assets, which is crucial in modern risk management. In Chapter 3, we approximate strongly coupled FBSDEs with a sequence of neural networks. More specifically, we are concerend with high-dimensional FBSDEs stemming from stochastic optimal control problems. These equations are inherently difficult to approximate due the coupling, and most classical methods as well as modern day machine learning algorithms break down. Our approach is novel in the sense that it combines mathematical structure from the initial control problem with the associate FBSDE which is shown to be both robust and accurate for a wide range of problems. In the final chapter, we consider time-inconsistent portfolio optimization problems. In general, for time-inconsistent problems the dynamic programming principle does not hold and in turn, the global problem cannot be divided into local sub-problems. Instead, we take on a fully data driven neural network-based approach to approximate the optimal allocation strategy directly. Beyond the classical instruments stocks and bonds, we also allow the investor to buy and sell options. In this way, we increase the flexibility in shaping the terminal wealth distribution.

show abstract

Learning Locomotion Controllers for Walking Using Deep FBSDE

Cited by 2 publications

References 15 publications

Convergence of a robust deep FBSDE method for stochastic control

Convergence of a robust deep FBSDE method for stochastic control

Neural networks for stochastic control and decision making in mathematical finance

Contact Info

Product

Resources

About