Management of Voltage Flexibility from Inverter-Based Distributed Generation Using Multi-Agent Reinforcement Learning

Abstract

The increase in the use of converter-interfaced generators (CIGs) in today’s electrical grids will require these generators both to supply power and participate in voltage control and provision of grid stability. At the same time, new possibilities of secondary QU droop control in power grids with a large proportion of CIGs (PV panels, wind generators, micro-turbines, fuel cells, and others) open new ways for DSO to increase energy flexibility and maximize hosting capacity. This study extends the existing secondary QU droop control models to enhance the efficiency of CIG integration into electrical networks. The paper presents an approach to decentralized control of secondary voltage through converters based on a multi-agent reinforcement learning (MARL) algorithm. A procedure is also proposed for analyzing hosting capacity and voltage flexibility in a power grid in terms of secondary voltage control. The effectiveness of the proposed static MARL control is demonstrated by the example of a modified IEEE 34-bus test feeder containing CIGs. Experiments have shown that the decentralized approach at issue is effective in stabilizing nodal voltage and preventing overcurrent in lines under various heavy load conditions often caused by active power injections from CIGs themselves and power exchange processes within the TSO/DSO market interaction.

Assessment of the energy systems resilience using artificial intelligence methods

“…In the energy industry, machine learning methods are used to solve a number of problems described, in particular, in [11][12][13][14][15][16][17][18]. The task of forecasting electricity consumption using machine learning models aims to improve energy efficiency and therefore manage energy consumption and ensure resilience.…”

DER Control and Management Strategies for Distribution Networks: A Review of Current Practices and Future Directions

“…The use of cellular 5G in the MAS-based control of DERs is one application area under investigation. Other research studies have combined multi-agent theory and reinforcement learning, which enables model-free, agent-based DER control [106]. A particular application focus is the use of distributed algorithms for voltage regulation in PV-rich distribution networks, and [42] utilises the distributed proximal atomic coordination (PAC) algorithm, while [106] develops a distributed voltage control scheme based on multi-agent reinforcement learning (MARL).…”

“…Addressing all of the possible power quality problems that can arise in DERs can be framed as a complex multi-objective optimisation. In this regard, several authors have proposed hierarchical control schemes based on advanced algorithms for constrained optimisation, such as genetic algorithms (GAs) [6, [34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51][91][92][93][94][95][96][98][99][100][101][102][103][104][105][106][107][108][109][110]. However, these are computationally intensive methods that require input data such as line and load parameters, inverter specifications, and information on the constraints and operating costs.…”

A Multi-Agent Reinforcement Learning Method for Cooperative Secondary Voltage Control of Microgrids

“…(5) Reward function: A well-designed reward function provides optimal guidance for learning the best strategy. Reference [37] only uses v odi to calculate the reward function. Although the voltage of DG can be adjusted, the bus voltage may still exceed the normal range.…”