Verification of Dissipativity and Evaluation of Storage Function in Economic Nonlinear MPC using Q-Learning

Kordabad, Arash Bahari; Gros, Sébastien

doi:10.1016/j.ifacol.2021.08.562

Cited by 6 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remark 3. In (19) the terminal cost is modelled with an FNN that does not preserve convexity. To ensure nominal stability the terminal cost should be at least positive definite.…”

Section: A Regular Neural Networkmentioning

confidence: 99%

“…Dissipativity of a problem is verified through the existence of a storage function that satisfies the dissipativity inequality [18]. Finding a valid storage function for the general problem is hard, but may be captured using RL techniques as suggested in [4] and justified in [19]. As we are focusing on economic problems, we use deterministic systems as a proof of concept, for which general dissipativity theory is valid.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Convex Neural Network-Based Cost Modifications for Learning Model Predictive Control

Seel

Kordabad

Gros

et al. 2022

IEEE Open J. Control. Syst.

View full text Add to dashboard Cite

Developing model predictive control (MPC) schemes can be challenging for systems where an accurate model is not available, or too costly to develop. With the increasing availability of data and tools to treat them, learning-based MPC has of late attracted wide attention. It has recently been shown that adapting not only the MPC model, but also its cost function is conducive to achieving optimal closed-loop performance when an accurate model cannot be provided. In the learning context, this modification can be performed via parametrizing the MPC cost and adjusting the parameters via, e.g., reinforcement learning (RL). In this framework, simple cost parametrizations can be effective, but the underlying theory suggests that rich parametrizations in principle can be useful. In this paper, we propose such a cost parametrization using a class of neural networks (NNs) that preserves convexity. This choice avoids creating difficulties when solving the MPC problem via sensitivity-based solvers. In addition, this choice of cost parametrization ensures nominal stability of the resulting MPC scheme. Moreover, we detail how this choice can be applied to economic MPC problems where the cost function is generic and therefore does not necessarily fulfill any specific property.

show abstract

“…Remark 3. In (19) the terminal cost is modelled with an FNN that does not preserve convexity. To ensure nominal stability the terminal cost should be at least positive definite.…”

Section: A Regular Neural Networkmentioning

confidence: 99%

mentioning

confidence: 99%

Convex Neural Network-Based Cost Modifications for Learning Model Predictive Control

Seel

Kordabad

Gros

et al. 2022

IEEE Open J. Control. Syst.

View full text Add to dashboard Cite

show abstract

Q-learning of the storage function in Economic Nonlinear Model Predictive Control

Kordabad

Gros

2022

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

Multi-agent Battery Storage Management using MPC-based Reinforcement Learning

Kordabad

Cai²,

Gros³

2021

2021 IEEE Conference on Control Technology and Applications (CCTA)

Self Cite

View full text Add to dashboard Cite

In this paper, we present the use of Model Predictive Control (MPC) based on Reinforcement Learning (RL) to find the optimal policy for a multi-agent battery storage system. A time-varying prediction of the power price and production-demand uncertainty are considered. We focus on optimizing an economic objective cost while avoiding very low or very high state of charge, which can damage the battery. We consider the bounded power provided by the main grid and the constraints on the power input and state of each agent. A parametrized MPC-scheme is used as a function approximator for the deterministic policy gradient method and RL optimizes the closed-loop performance by updating the parameters. Simulation results demonstrate that the proposed method is able to tackle the constraints and deliver the optimal policy.

show abstract

Verification of Dissipativity and Evaluation of Storage Function in Economic Nonlinear MPC using Q-Learning

Cited by 6 publications

References 18 publications

Convex Neural Network-Based Cost Modifications for Learning Model Predictive Control

Convex Neural Network-Based Cost Modifications for Learning Model Predictive Control

Q-learning of the storage function in Economic Nonlinear Model Predictive Control

Multi-agent Battery Storage Management using MPC-based Reinforcement Learning

Contact Info

Product

Resources

About