Sequential alternating least squares for solving high dimensional linear Hamilton-Jacobi-Bellman equation

Stefansson, Elis; Leong, Yoke Peng

doi:10.1109/iros.2016.7759553

Cited by 20 publications

(15 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Iteration in policy space for second order HJB PDEs arising in stochastic control is discussed in [41]. Tensor calculus technique is used in [40] where the associated HJB equation becomes linear after using some exponential transformation and scaling factor. For an overview for numerical approximations to stochastic HJB equations, we refer to [31].…”

Section: Introductionmentioning

confidence: 99%

Policy iteration for Hamilton–Jacobi–Bellman equations with control constraints

Kundu

Kunisch

2021

Comput Optim Appl

View full text Add to dashboard Cite

Policy iteration is a widely used technique to solve the Hamilton Jacobi Bellman (HJB) equation, which arises from nonlinear optimal feedback control theory. Its convergence analysis has attracted much attention in the unconstrained case. Here we analyze the case with control constraints both for the HJB equations which arise in deterministic and in stochastic control cases. The linear equations in each iteration step are solved by an implicit upwind scheme. Numerical examples are conducted to solve the HJB equation with control constraints and comparisons are shown with the unconstrained cases.

show abstract

Section: Introductionmentioning

confidence: 99%

Policy iteration for Hamilton–Jacobi–Bellman equations with control constraints

Kundu

Kunisch

2021

Comput Optim Appl

View full text Add to dashboard Cite

show abstract

“…SOS-based controller synthesis methods are presented in [4], [5] and [6] for the continuous-time (CT) and discrete-time (DT) settings, respectively. Examples of computational methods based on approximating solutions of Hamilton-Jacobi-Bellman type of equations are found in [7], [8], [9]. Other, more recent works [10] (CT), [11] (DT) provide alternative formulations of optimal controller synthesis through occupation measures.…”

Section: Introductionmentioning

confidence: 99%

A System Level Approach to Discrete-Time Nonlinear Systems

2020

2020 American Control Conference (ACC)

View full text Add to dashboard Cite

We will show that there is a universal connection between the achievable closed-loop dynamics and the corresponding feedback controller that produces it. This connection shows promise to lead to new methods for robust nonlinear control in discrete-time. We will show that, given a causal nonlinear discrete-time system and controller, the resulting closed-loop is a solution to a nonlinear operator equation. Conversely, any causal solution to the nonlinear operator equation is a closedloop that can be achieved by some causal controller. Moreover, solutions can be substituted into a simple dynamic controller structure, which we will refer to as a system level controller, to obtain an implementation of the unique corresponding feedback controller. System level controllers could be an attractive approach for robust nonlinear control, as we will show that even when they are parametrized with approximate solutions to the operator equation, they can still produce robustly stable closed loops. We will provide theoretical results that state how grade of approximation and robust stability of the closed loop are related. Additionally, we will explore some first applications of our results. Using the cart-pole system as an illustrative example, we will derive how to design robust discrete-time trajectory tracking controllers for continuous-time nonlinear systems. Secondly, we will introduce a particular class of system level controller that shows to be particularly useful for linear systems with actuator saturation and state constraints; The special structure of the controller allows for simple stability and performance analysis of the closed-loop in presence of disturbances. The structure also offers simple ways to do antiwindup compensation, and provides a new nonlinear approach to the constrained LQG problem. A particular application to large-scale systems with actuator saturation and safety constraints is presented in our companion paper [1].

show abstract

“…However, the design of numerical methods for the solution of high-dimensional HJ PDEs remains a daunting task. Along this direction, some encouraging results have been obtained over the last years in connection with the use of sparse grids [11,29], causality-free methods [34,22,48], machine learning techniques [40,39], tensor calculus [47], graph-tree structures [3], Taylor series expansions [17], and polynomial approximation [32]. In this latter work, we develop a numerical scheme based on a high-dimensional polynomial ansatz for the value function coupled with a Newton-type (policy iteration) method for the solution of the Galerkin residual equation associated to the HJB equation.…”

Section: Introductionmentioning

confidence: 99%

Robust Feedback Control of Nonlinear PDEs by Numerical Approximation of High-Dimensional Hamilton--Jacobi--Isaacs Equations

Kalise¹,

Kundu²,

Kunisch³

2020

SIAM J. Appl. Dyn. Syst.

View full text Add to dashboard Cite

We propose an approach for the synthesis of robust and optimal feedback controllers for nonlinear PDEs. Our approach considers the approximation of infinite-dimensional control systems by a pseudospectral collocation method, leading to high-dimensional nonlinear dynamics. For the reduced-order model, we construct a robust feedback control based on the H∞ control method, which requires the solution of an associated high-dimensional Hamilton-Jacobi-Isaacs nonlinear PDE. The dimensionality of the Isaacs PDE is tackled by means of a separable representation of the control system, and a polynomial approximation ansatz for the corresponding value function. Our method proves to be effective for the robust stabilization of nonlinear dynamics up to dimension d ≈ 12. We assess the robustness and optimality features of our design over a class of nonlinear parabolic PDEs, including nonlinear advection and reaction terms. The proposed design yields a feedback controller achieving optimal stabilization and disturbance rejection properties, along with providing a modelling framework for the robust control of PDEs under parametric uncertainties.

show abstract

Sequential alternating least squares for solving high dimensional linear Hamilton-Jacobi-Bellman equation

Cited by 20 publications

References 16 publications

Policy iteration for Hamilton–Jacobi–Bellman equations with control constraints

Policy iteration for Hamilton–Jacobi–Bellman equations with control constraints

A System Level Approach to Discrete-Time Nonlinear Systems

Robust Feedback Control of Nonlinear PDEs by Numerical Approximation of High-Dimensional Hamilton--Jacobi--Isaacs Equations

Contact Info

Product

Resources

About