Stochastic <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="mml5" display="inline" overflow="scroll" altimg="si5.gif"><mml:msup><mml:mrow><mml:mi>L</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:math>-optimal control via forward and backward sampling

Exarchos, Ioannis; Theodorou, Evangelos A.; Tsiotras, Panagiotis

doi:10.1016/j.sysconle.2018.06.005

Cited by 30 publications

(37 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is easy to verify that the PDE associated with the new system is the same as the original one (12). For the full derivation of change of measure for FBSDEs, we refer readers to proof of Theorem 1 in [14].…”

Section: Importance Samplingmentioning

confidence: 99%

Deep Forward-Backward SDEs for Min-max Control

Wang

Lee

Pereira

et al. 2019

2019 IEEE 58th Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

This paper presents a novel approach to numerically solve stochastic differential games for nonlinear systems. The proposed approach relies on the nonlinear Feynman-Kac theorem that establishes a connection between parabolic deterministic partial differential equations and forward-backward stochastic differential equations. Using this theorem the Hamilton-Jacobi-Isaacs partial differential equation associated with differential games is represented by a system of forwardbackward stochastic differential equations. Numerical solution of the aforementioned system of stochastic differential equations is performed using importance sampling and a Long-Short Term Memory recurrent neural network, which is trained in an offline fashion. The resulting algorithm is tested on two example systems in simulation and compared against the standard risk neutral stochastic optimal control formulations.

show abstract

Section: Importance Samplingmentioning

confidence: 99%

Deep Forward-Backward SDEs for Min-max Control

Wang

Lee

Pereira

et al. 2019

2019 IEEE 58th Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Such control constraints are common in mechanical systems, where control forces and/or torques are bounded, and may be readily introduced in our framework via the addition of a "soft" constraint, integrated within the cost functional. In recent work, Exarchos et al [26] showed how box-type control constraints for L 1 -optimal control problems (also called minimum fuel problems), can be incorporated into an FBSDE scheme. These are in contrast to the more frequently used quadratic control cost (L 2 or minimum energy) SOC problems.…”

Section: Fbsde Reformulationmentioning

confidence: 99%

“…The initial valueỹ i 0 and its gradientz i 0 are parameterized by trainable variables φ and are randomly initialized. The optimal control action is calculated using the discretized version of (6) (or (26) for the control constrained case). The dynamicsx and value functionỹ are propagated using the Euler integration scheme, as shown in the algorithm.…”

Section: Deep Fbsde Controllermentioning

confidence: 99%

“…Therefore, numerical approximation techniques are needed for utilization in an actual algorithm. Exarchos and Theodorou [14] developed an importance sampling based iterative scheme by approximating the conditional expectation at every time step using linear regression (see also [15] and [16]). However, this method suffers from compounding errors from Least Squares approximation at every time step.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Deep Stochastic Optimal Control Policies Using Forward-Backward SDEs

Pereira¹,

Wang²,

Theodorou³

2019

Robotics: Science and Systems XV

View full text Add to dashboard Cite

In this paper we propose a new methodology for decision-making under uncertainty using recent advancements in the areas of nonlinear stochastic optimal control theory, applied mathematics, and machine learning. Grounded on the fundamental relation between certain nonlinear partial differential equations and forward-backward stochastic differential equations, we develop a control framework that is scalable and applicable to general classes of stochastic systems and decision-making problem formulations in robotics and autonomy. The proposed deep neural network architectures for stochastic control consist of recurrent and fully connected layers. The performance and scalability of the aforementioned algorithm are investigated in three non-linear systems in simulation with and without control constraints. We conclude with a discussion on future directions and their implications to robotics.

show abstract

“…Usual difficulties involve the solution of the Hamilton-Jacobi-Bellman (HJB) equations associated to the optimal control problem: in [5], the stochastic HJB equation is iteratively solved with successive approximations; in [6], the infinitetime HJB equation is reformulated as an eigenvalue problem; in [7], a transformation approach is proposed for solving the HJB equation arising in quadratic-cost control for nonlinear deterministic and stochastic systems. Finally, in a pair of recent papers, a solution to the nonlinear HJB equation is provided, by expressing it in the form of decoupled Forward and Backward Stochastic Differential Equations (FBSDEs), for an L 2 -and an L 1 -type optimal control setting (see [8,9], respectively). As stated above, the solutions proposed in these references rely on a complete knowledge of the state of the system; thus, they do not require any nonlinear stateestimation algorithm to infer information from noisy measurements.…”

Section: Introductionmentioning

confidence: 99%

A Stochastic Optimal Regulator for a Class of Nonlinear Systems

Mavelli

Palombo

Palumbo

2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

This work investigates an optimal control problem for a class of stochastic differential bilinear systems, affected by a persistent disturbance provided by a nonlinear stochastic exogenous system (nonlinear drift and multiplicative state noise). The optimal control problem aims at minimizing the average value of a standard quadratic-cost functional on a finite horizon. It has been supposed that neither the state of the system nor the state of the exosystem is directly measurable (incomplete information case). The approach is based on the Carleman embedding, which allows to approximate the nonlinear stochastic exosystem in the form of a bilinear system (linear drift and multiplicative noise) with respect to an extended state that includes the state Kronecker powers up to a chosen degree. This way the stochastic optimal control problem may be restated in a bilinear setting and the optimal solution is provided among all the affine transformations of the measurements. The present work is a nontrivial extension of previous work of the authors, where the Carleman approach was exploited in a framework where only additive noises had been conceived for the state and for the exosystem. Numerical simulations support theoretical results by showing the improvements in the regulator performances by increasing the order of the approximation.

show abstract

Stochastic L1-optimal control via forward and backward sampling

Cited by 30 publications

References 13 publications

Deep Forward-Backward SDEs for Min-max Control

Deep Forward-Backward SDEs for Min-max Control

Learning Deep Stochastic Optimal Control Policies Using Forward-Backward SDEs

A Stochastic Optimal Regulator for a Class of Nonlinear Systems

Contact Info

Product

Resources

About