Adaptive dynamic programming and deep reinforcement learning for the control of an unmanned surface vehicle: Experimental results

González-García, Alejandro; Barragan-Alcantar, David; Collado-Gonzalez, Ivana; Garrido, Leonardo

doi:10.1016/j.conengprac.2021.104807

Cited by 35 publications

(11 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the resulting optimization problem cannot be solved analytically in the presence of constraints, and thus, numerical methods are needed. RL-based controllers provide alternative solutions to MPC either relying or without relying on models to control the process [26]- [31]. In the model-free setting, an agent interacts with the process by taking the best actions given the current states.…”

Section: A Model Predictive Control (Mpc) and Reinforcement Learning ...mentioning

confidence: 99%

Skew Filtering for Online State Estimation and Control

DOGRU¹,

Chiplunkar²,

Huang³

2022

Preprint

View full text Add to dashboard Cite

<div>Process optimization and control can become challenging when the measurements are affected by irregular noise. Classical approaches utilize Gaussian method like Kalman filtering to alleviate the sensory noise. However, many industries involve skewed noise in their processes. While the closed skew normal (CSN) distribution generalizes a Gaussian distribution with additional parameters, its dimension increases during recursive estimation, making it impractical. Even though there exist some techniques for the solution, they are typically either too complicated or inaccurate for the higher dimensional problems. This study proposes a novel online optimization scheme to reduce the dimensionality of a CSN distribution while considering the properties of the complete empirical distribution. Since the objective function used during the optimization step considers the geometry of the metric space, the proposed scheme achieves the higher accuracy without sacrificing computational efficiency. After finding the reliable combination of objective function and optimizer, the proposed filter is applied to two real-time pilot-scale experiments. The results indicate that it is beneficial for recursive state estimation in the presence of skewed noise.<br></div>

show abstract

Section: A Model Predictive Control (Mpc) and Reinforcement Learning ...mentioning

confidence: 99%

Skew Filtering for Online State Estimation and Control

DOGRU¹,

Chiplunkar²,

Huang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Obviously, the initial admissible control policy for nonlinear systems is difficult to obtain, which means that VI has a high application value if the convergence rate of VI can be boosted. ADP has many success examples in areas such as smart home, 20 power grid, 21 and aircraft control 22,23 . Currently, ADP has become a critical class of methods for solving optimal control policies for nonlinear systems.…”

Section: Introductionmentioning

confidence: 99%

“…ADP has many success examples in areas such as smart home, 20 power grid, 21 and aircraft control. 22,23 Currently, ADP has become a critical class of methods for solving optimal control policies for nonlinear systems.…”

Section: Introductionmentioning

confidence: 99%

Adaptive critic control with multi‐step policy evaluation for nonlinear zero‐sum games

Li,

Wang,

Wang

et al. 2023

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

To attenuate the effect of disturbances on control performance, a multi‐step adaptive critic control (MsACC) framework is developed to solve zero‐sum games for discrete‐time nonlinear systems. The MsACC algorithm utilizes multi‐step policy evaluation to obtain the solution of the Hamilton–Jacobi–Isaac equation, which is faster than that of the one‐step policy evaluation. The convergence rate of the MsACC algorithm is adjustable by varying the step size of the policy evaluation. In addition, the stability and convergence of the MsACC algorithm are proved under certain conditions. In order to realize the MsACC algorithm, three neural networks are established to approximate the control input, the disturbance input, and the cost function, respectively. Finally, the effectiveness of the MsACC algorithm is verified by two simulation examples, including a linear system and a nonlinear plant.

show abstract

“…Great effort has been made to developing ADP algorithms for discrete‐time systems, 16,17 as well as continuous‐time systems 18‐20 . In practical applications, ADP algorithms have been used broadly in many fields 21‐28 …”

Section: Introductionmentioning

confidence: 99%

Differential dynamic programming for finite‐horizon zero‐sum differential games of nonlinear systems

Zhang,

Jia,

Zhang

2023

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

In this article, we present an iterative algorithm based on differential dynamic programming (DDP) for finite‐horizon two‐person zero‐sum differential games. The technique of DDP is used to expand the Hamilton–Jacobi–Isaacs (HJI) partial differential equation into higher‐order differential equations. Using value function and saddle point approximations, the DDP expansion is transformed into algebraic matrix equation in integral form. Based on the algebraic matrix equation, a DDP iterative algorithm is developed to learn the solution to the differential games. Strict proof is proposed to guarantee the iterative convergences of the value function and saddle point. The new algorithm is fundamentally different from existing results, in the sense that it overcome the technical obstacle to address the time‐varying behavior of HJI partial differential equation. Simulation examples are given to demonstrate the effectiveness of the proposed method.

show abstract

Adaptive dynamic programming and deep reinforcement learning for the control of an unmanned surface vehicle: Experimental results

Cited by 35 publications

References 30 publications

Skew Filtering for Online State Estimation and Control

Skew Filtering for Online State Estimation and Control

Adaptive critic control with multi‐step policy evaluation for nonlinear zero‐sum games

Differential dynamic programming for finite‐horizon zero‐sum differential games of nonlinear systems

Contact Info

Product

Resources

About