Reinforcement Learning based on Scenario-tree MPC for ASVs

Abstract

In this paper, we present the use of Reinforcement Learning (RL) based on Robust Model Predictive Control (RMPC) for the control of an Autonomous Surface Vehicle (ASV). The RL-MPC strategy is utilized for obstacle avoidance and target (set-point) tracking. A scenario-tree robust MPC is used to handle potential failures of the ship thrusters. Besides, the wind and ocean current are considered as unknown stochastic disturbances in the real system, which are handled via constraints tightening. The tightening and other cost parameters are adjusted by RL, using a Q-learning technique. An economic cost is considered, minimizing the time and energy required to achieve the ship missions. The method is illustrated in simulation on a nonlinear 3-DOF model of a scaled version of the Cybership II.

Automatic tuning of robust model predictive control in iterative tasks using efficient Bayesian optimization

“…Model predictive control (MPC) has attained remarkable success in recent decades, because of its disturbance rejection (Draeger et al, 1995) and constraint handling (Morari and Lee, 1999) capabilities. It has been widely applied in various fields (Darby and Nikolaou, 2012), such as robotics, process control and reinforcement learning (Chua et al, 2018; Kordabad et al, 2021; Pfrommer et al, 2022). However, the performance of the MPC controllers can be degraded by a series of factors, including an uncertain system model (Piga et al, 2019), a limited terminal set (Rosolia and Borrelli, 2017, 2018), or an inappropriate objective function (Marco et al, 2016).…”