Deep-Reinforcement-Learning-Based Two-Timescale Voltage Control for Distribution Systems

Abstract

Because of the high penetration of renewable energies and the installation of new control devices, modern distribution networks are faced with voltage regulation challenges. Recently, the rapid development of artificial intelligence technology has introduced new solutions for optimal control problems with high dimensions and dynamics. In this paper, a deep reinforcement learning method is proposed to solve the two-timescale optimal voltage control problem. All control variables are assigned to different agents, and discrete variables are solved by a deep Q network (DQN) agent while the continuous variables are solved by a deep deterministic policy gradient (DDPG) agent. All agents are trained simultaneously with specially designed reward aiming at minimizing long-term average voltage deviation. Case study is executed on a modified IEEE-123 bus system, and the results demonstrate that the proposed algorithm has similar or even better performance than the model-based optimal control scheme and has high computational efficiency and competitive potential for online application.

Reform and Innovation of Civic and Political Education Based on Deep Enhanced Learning Methods

“…Deep reinforcement learning provides the idea for solving the perceptual decision problem of complex systems [19][20]. In this paper, we will study the innovation method of the reform and reform of education reform.…”

Review of Deep Reinforcement Learning and Its Application in Modern Renewable Power System Control

“…The latter applies the DQN algorithm to determine the reactive power control strategy of switching the shunt capacitors in a slow timescale (e.g., hours, days). Moreover, the author of [35] applies hybrid DRL algorithms to different timescale control devices to generate optimal control policy in both fast and slow timescales with the continuous and discrete domain. In particular, multiple agents are divided into DQN-based and DDPGbased agents, which are used for discrete actions, including the configuration of capacitors in a slow timescale and continuous actions, such as the control strategies of PV inverters and energy storage batteries, in a fast timescale.…”

Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm

“…This method has good robustness and does not depend on communication technology. In contrast to the method of Shuang et al (2021), Zhang et al (2021) adopted the DQN algorithm and DDPG algorithm. The DQN-DDPG algorithm was employed in this paper, but we also considered whether the DGs and the reactive voltage regulation equipment were connected as variables for optimizing the active and reactive power.…”

“…The DQN-DDPG algorithm was employed in this paper, but we also considered whether the DGs and the reactive voltage regulation equipment were connected as variables for optimizing the active and reactive power. Zhang et al (2021) did not take into account the effects of active DPV power reduction on voltage regulation of the DN. Liu et al, (2021) and Zhou et al (2021) proposed a scheduling scheme for an ES system on a DN based on deep reinforcement learning with high permeability DPV access to reduce voltage deviations.…”