Configuration of the Actor and Critic Network of the Deep Reinforcement Learning controller for Multi-Energy Storage System

Abstract

The computational burden and the time required to train a deep reinforcement learning (DRL) can be appreciable, especially for the particular case of a DRL control used for frequency control of multi-electrical energy storage (MEESS). This paper presents an assessment of four training configurations of the actor and critic network to determine the configuration training that produces the lower computational time, considering the specific case of frequency control of MEESS. The training configuration cases are defined considering two processing units: CPU and GPU and are evaluated considering serial and parallel computing using MATLAB ® 2020b Parallel Computing Toolbox. The agent used for this assessment is the Deep Deterministic Policy Gradient (DDPG) agent. The environment represents the dynamic model to provide enhanced frequency response to the power system by controlling the state of charge of energy storage systems. Simulation results demonstrated that the best configuration to reduce the computational time is training both actor and critic network on CPU using parallel computing.

Secure and Smart: Enhancing Energy Systems in Core Electrical Networks

“…Cyberattacks on smart grids can have very bad results, such as service interruptions, financial losses, and even threats to public safety. So, strong security measures must be built into smart grid systems to protect important assets and make sure that energy delivery is always reliable [1].This paper suggests a complete plan for improving energy systems in main power grids, with a focus on incorporating smart technologies and safety measures. The framework is meant to deal with the problems that current energy systems have, such as the need for better safety, dependability, and economy.…”