Deep Learning Optimal Control for a Complex Hybrid Energy Storage System

Abstract

Deep Reinforcement Learning (DRL) proved to be successful for solving complex control problems and has become a hot topic in the field of energy systems control, but for the particular case of thermal energy storage (TES) systems, only a few studies have been reported, all of them with a complexity degree of the TES system far below the one of this study. In this paper, we step forward through a DRL architecture able to deal with the complexity of an innovative hybrid energy storage system, devising appropriate high-level control operations (or policies) over its subsystems that result optimal from an energy or monetary point of view. The results show that a DRL policy in the system control can reduce the system operating costs by more than 50%, as compared to a rule-based control (RBC) policy, for cooling supply to a reference residential building in Mediterranean climate during a period of 18 days. Moreover, a robustness analysis was carried out, which showed that, even for large errors in the parameters of the system simulation models corresponding to an error multiplying factors up to 2, the average cost obtained with the original model deviates from the optimum value by less than 3%, demonstrating the robustness of the solution over a wide range of model errors.

Strategies for Controlling Microgrid Networks with Energy Storage Systems: A Review

“…Additionally, the main goal of reducing carbon impact was achieved when the efficiency of the hydrogen storage was adequately large. The smart deep RL-based strategy that was proposed in [119] was the most recent distinctively successful attempt to control a complex hybrid electrical and thermal storage system that was fed by a PV system of a residential building. The main aim of the new strategy was to reduce energy obliged for heating, cooling, and providing hot water.…”

Deep Reinforcement Learning for Autonomous Water Heater Control

“…Al-jabery et al [28] proposed a Q-learning-based approach to solving the multi-objective optimization problem of minimizing the total cost of the power consumed, reducing the power demand during the peak period, and achieving customer satisfaction. Zsembinszki et al [29] developed an RL-based controller to reduce the energy demand for heating, cooling and domestic hot water and compared the RL-based controller with a rule-based controller.…”

Effective estimation of the state-of-charge of latent heat thermal energy storage for heating and cooling systems using non-linear state observers

“…The adoption of TES units requires implementing effective control strategies to regulate their operation. For instance, model predictive control has been employed to analyse the optimal charging period of thermal stores while reducing operational costs and achieving thermal comfort [16,17]. On the other hand, optimisation algorithms have been https://doi.org/10.1016/j.apenergy.…”