SympOCnet: Solving optimal control problems with applications to high-dimensional multi-agent path planning problems

Meng, Tingwei; Zhang, Zhen; Darbon, Jérôme; Karniadakis, George

doi:10.48550/arxiv.2201.05475

Cited by 4 publications

(2 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…structures whose size grow exponentially with the problem dimension. A second group of methods attempt to solve the HJB PDE in the least-squares sense by minimizing the residual of the PDE and boundary conditions at (randomly sampled) collocation points [18]- [21]. More recently, [22]- [24] have proposed methods to solve the HJB equation along its characteristics without generating data.…”

Section: Introductionmentioning

confidence: 99%

Neural Network Optimal Feedback Control With Guaranteed Local Stability

Nakamura-Zimmerer

Gong

Kang

2022

IEEE Open J. Control. Syst.

View full text Add to dashboard Cite

Recent research shows that supervised learning can be an effective tool for designing nearoptimal feedback controllers for high-dimensional nonlinear dynamic systems. But the behavior of neural network controllers is still not well understood. In particular, some neural networks with high test accuracy can fail to even locally stabilize the dynamic system. To address this challenge we propose several novel neural network architectures, which we show guarantee local asymptotic stability while retaining the approximation capacity to learn the optimal feedback policy semi-globally. The proposed architectures are compared against standard neural network feedback controllers through numerical simulations of two high-dimensional nonlinear optimal control problems: stabilization of an unstable Burgers-type partial differential equation, and altitude and course tracking for an unmanned aerial vehicle. The simulations demonstrate that standard neural networks can fail to stabilize the dynamics even when trained well, while the proposed architectures are always at least locally stabilizing. Moreover, the proposed controllers are found to be close to optimal in testing.INDEX TERMS Computational methods, machine learning and control, neural networks, nonlinear control systems, optimal control.

show abstract

Section: Introductionmentioning

confidence: 99%

Neural Network Optimal Feedback Control With Guaranteed Local Stability

Nakamura-Zimmerer

Gong

Kang

2022

IEEE Open J. Control. Syst.

View full text Add to dashboard Cite

show abstract

“…Unfortunately, for the fully nonlinear setting, no reformulation is possible and the HJB PDE must be solved directly. In this direction, over the last years there has been a significant progress on the solution of high-dimensional HJB PDEs arising in optimal control, including max-plus algebra methods [34,1,11], sparse grids [18], tree-structure algorithms [4] and applications of artificial neural networks [25,12,33,35,49,38]. The above mentioned techniques can scale up to very high-dimensional HJB PDEs, however, the effective implementation of real-time HJB-based controllers remains an open problem.…”

mentioning

confidence: 99%

Data-driven Tensor Train Gradient Cross Approximation for Hamilton-Jacobi-Bellman Equations

Dolgov¹,

Kalise²,

Saluzzi³

2022

Preprint

View full text Add to dashboard Cite

A gradient-enhanced functional tensor train cross approximation method for the resolution of the Hamilton-Jacobi-Bellman (HJB) equations associated to optimal feedback control of nonlinear dynamics is presented. The procedure uses samples of both the solution of the HJB equation and its gradient to obtain a tensor train approximation of the value function. The collection of the data for the algorithm is based on two possible techniques: Pontryagin Maximum Principle and State-Dependent Riccati Equations. Several numerical tests are presented in low and high dimension showing the effectiveness of the proposed method and its robustness with respect to inexact data evaluations, provided by the gradient information. The resulting tensor train approximation paves the way towards fast synthesis of the control signal in real-time applications.

show abstract

A Multilinear HJB-POD Method for the Optimal Control of PDEs on a Tree Structure

Kirsten,

Saluzzi

2024

J Sci Comput

View full text Add to dashboard Cite

Optimal control problems driven by evolutionary partial differential equations arise in many industrial applications and their numerical solution is known to be a challenging problem. One approach to obtain an optimal feedback control is via the Dynamic Programming principle. Nevertheless, despite many theoretical results, this method has been applied only to very special cases since it suffers from the curse of dimensionality. Our goal is to mitigate this crucial obstruction developing a version of dynamic programming algorithms based on a tree structure and exploiting the compact representation of the dynamical systems based on tensors notations via a model reduction approach. Here, we want to show how this algorithm can be constructed for general nonlinear control problems and to illustrate its performances on a number of challenging numerical tests introducing novel pruning strategies that improve the efficacy of the method. Our numerical results indicate a large decrease in memory requirements, as well as computational time, for the proposed problems. Moreover, we prove the convergence of the algorithm and give some hints on its implementation.

show abstract

SympOCnet: Solving optimal control problems with applications to high-dimensional multi-agent path planning problems

Cited by 4 publications

References 71 publications

Neural Network Optimal Feedback Control With Guaranteed Local Stability

Neural Network Optimal Feedback Control With Guaranteed Local Stability

Data-driven Tensor Train Gradient Cross Approximation for Hamilton-Jacobi-Bellman Equations

A Multilinear HJB-POD Method for the Optimal Control of PDEs on a Tree Structure

Contact Info

Product

Resources

About