Convex Neural Network-Based Cost Modifications for Learning Model Predictive Control

Abstract

Developing model predictive control (MPC) schemes can be challenging for systems where an accurate model is not available, or too costly to develop. With the increasing availability of data and tools to treat them, learning-based MPC has of late attracted wide attention. It has recently been shown that adapting not only the MPC model, but also its cost function is conducive to achieving optimal closed-loop performance when an accurate model cannot be provided. In the learning context, this modification can be performed via parametrizing the MPC cost and adjusting the parameters via, e.g., reinforcement learning (RL). In this framework, simple cost parametrizations can be effective, but the underlying theory suggests that rich parametrizations in principle can be useful. In this paper, we propose such a cost parametrization using a class of neural networks (NNs) that preserves convexity. This choice avoids creating difficulties when solving the MPC problem via sensitivity-based solvers. In addition, this choice of cost parametrization ensures nominal stability of the resulting MPC scheme. Moreover, we detail how this choice can be applied to economic MPC problems where the cost function is generic and therefore does not necessarily fulfill any specific property.

Variance-Based Exploration for Learning Model Predictive Control

“…We note that rich function approximators, such as NNs, can be used with the proposed framework to capture complex functions in the MPC scheme. In [22], the authors suggest using convex NNs to modify the stage cost in (6a). Next, we will describe Q-learning which is one method that can be used to update the parameters θ.…”