An Online Learning Approach to Model Predictive Control

Abstract

Model predictive control (MPC) is a powerful technique for solving dynamic control tasks. In this paper, we show that there exists a close connection between MPC and online learning, an abstract theoretical framework for analyzing online decision making in the optimization literature. This new perspective provides a foundation for leveraging powerful online learning algorithms to design MPC algorithms. Specifically, we propose a new algorithm based on dynamic mirror descent (DMD), an online learning algorithm that is designed for non-stationary setups. Our algorithm, Dynamic Mirror Descent Model Predictive Control (DMD-MPC), represents a general family of MPC algorithms that includes many existing techniques as special instances. DMD-MPC also provides a fresh perspective on previous heuristics used in MPC and suggests a principled way to design new MPC algorithms. In the experimental section of this paper, we demonstrate the flexibility of DMD-MPC, presenting a set of new MPC algorithms on a simple simulated cartpole and a simulated and real-world aggressive driving task. Videos of the real-world experiments can be found at https://youtu.be/vZST3v0_S9w and https://youtu.be/MhuqiHo2t98.

Online Weight-adaptive Nonlinear Model Predictive Control

“…They considered multi-input multioutput systems that can be expressed by a finite impulse response model. In [16], a close connection between MPC and online learning was shown. The authors have proposed a new algorithm based on dynamic mirror descent.…”

“…Adaptive Model Predictive Control (MPC) was studied in [2], [15], [17], [18] and [16]. In [2], the authors proposed an adaptive multi-variable zone controller and gave robustness guarantees for the controlled process.…”

“…On the other hand, online and data-adaptive strategies for updating the cost weights during trajectory execution were not studied extensively. In this line, besides the link between NMPC and online learning [16], other connections to known machine learning paradigms with emphasis on the weigh estimation remain not explored.…”

Imitation learning for agile autonomous driving

“…Unlike quadratic programming (QP)-based approaches (Borrelli et al, 2005), our DDP-based approach is self-contained and does not rely on an external optimization solver. Compared with sampling-based method (Wagener et al, 2019; Williams et al, 2017) that uses massive forward simulations, our approach is more efficient as it exploits of the structure of the dynamics model (19).…”

When Your Robot Breaks: Active Learning During Plant Failure

“…Recent work has shown the potential for MPC to be used in conjunction with online learning techniques. [9] presents an online learning approach to designing model predictive controllers. This framework utilizes online learning techniques to learn the parameters that minimize the MPC objective.…”