Application of SARSA learning algorithm for reactive power control in power system

Tousi, M.R.; Hosseinian, Seyed Hossein; Jadidinejad, Amir Hossein; Menhaj, Mohammad Bagher

doi:10.1109/pecon.2008.4762658

Cited by 8 publications

(5 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A decentralized multiagent DRL framework that utilizes reactive power control of photovoltaics (PVs), and storage systems was proposed in paper [ 45 ]. Other RL reactive power optimizations were presented in papers [ 46 , 47 ]. Nevertheless, there is a lack of literature addressing various challenges, such as adequate formulation of the reward function and finding the optimal combination of hyperparameters that effectively leverage network architectures’ strengths and training procedures to maximize performance.…”

Section: Introductionmentioning

confidence: 99%

Optimal Reactive Power Dispatch in ADNs using DRL and the Impact of Its Various Settings and Environmental Changes

Zamzam,

Shaban,

Massoud

2023

Sensors

View full text Add to dashboard Cite

Modern active distribution networks (ADNs) witness increasing complexities that require efforts in control practices, including optimal reactive power dispatch (ORPD). Deep reinforcement learning (DRL) is proposed to manage the network’s reactive power by coordinating different resources, including distributed energy resources, to enhance performance. However, there is a lack of studies examining DRL elements’ performance sensitivity. To this end, in this paper we examine the impact of various DRL reward representations and hyperparameters on the agent’s learning performance when solving the ORPD problem for ADNs. We assess the agent’s performance regarding accuracy and training time metrics, as well as critic estimate measures. Furthermore, different environmental changes are examined to study the DRL model’s scalability by including other resources. Results show that compared to other representations, the complementary reward function exhibits improved performance in terms of power loss minimization and convergence time by 10–15% and 14–18%, respectively. Also, adequate agent performance is observed to be neighboring the best-suited value of each hyperparameter for the studied problem. In addition, scalability analysis depicts that increasing the number of possible action combinations in the action space by approximately nine times results in 1.7 times increase in the training time.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimal Reactive Power Dispatch in ADNs using DRL and the Impact of Its Various Settings and Environmental Changes

Zamzam,

Shaban,

Massoud

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Reference [14] applied DQN and Deep Deterministic Policy Gradient (DDPG) for subsystem voltage control and found that DDPG performed better with sufficient training scenarios. The voltage set point of a STATCOM is regulated using SARSA to facilitate discrete reactive power injection for voltage control in [20]. ESS, PV, and SVC output power levels are managed with the SAC algorithm to mitigate voltage violations in [21] where predefined discrete power levels are used for voltage control.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Excitation Control-Based Voltage Regulation Using DDPG Considering System Dynamic Performance

Wang,

Vittal

2023

IEEE Open J. Power Energy

View full text Add to dashboard Cite

In recent years, there has been an increasing need for effective voltage control methods in power systems due to the growing complexity and dynamic nature of practical power grid operations. This paper proposes a real-time voltage control method based on deep reinforcement learning (DRL) that continuously regulates the excitation system in response to system disturbances. Dynamic performance is considered during control by incorporating the voltage dynamics data that influence the practical power grid operation. The proposed approach utilizes the deep deterministic policy gradient (DDPG) algorithm, capable of handling continuous action spaces, to adjust the voltage reference of the generator excitation system in real time. To analyze the power system dynamic process, a versatile transmission-level power system dynamic training and simulation platform is developed by integrating the power system simulation software PSS/E and a user-written DRL agent code developed in Python. The platform facilitates the training and testing of various power system algorithms and power grids in dynamic simulations. The efficacy of the proposed method is evaluated based on the developed platform through extensive case studies on the IEEE 9-bus system and the Texas 2000-bus system. The results validate the effectiveness of the approach, highlighting its promising performance in real-time control with respect to dynamic processes.INDEX TERMS Voltage control, deep reinforcement learning, DDPG, power system dynamic control, real-time, excitation control.

show abstract

“…A reinforcement learning agent-in our case, a planning bot-gains decision-making knowledge by repetitively interacting with the surrounding environment (TPS) and evaluating rewards (improvement of the plan dose distribution) associated with the action (changing of optimization objectives). State-action-reward-state-action (SARSA) 7 , also known as connectionist Q-learning, is a widely-used reinforcement learning algorithm and has been proven to perform well in wide-ranging real-world applications such as controlling power systems 8 , advanced robotics 9 , and playing video games 10,11 . It is an efficient, sampling-based algorithm that sequentially changes the knowledge of the agent based on the interactive training process.…”

Section: Introductionmentioning

confidence: 99%

An Interpretable Planning Bot for Pancreas Stereotactic Body Radiation Therapy

Zhang

Wang

Yang

et al. 2021

International Journal of Radiation Oncology*Biology*Physics

View full text Add to dashboard Cite

Application of SARSA learning algorithm for reactive power control in power system

Cited by 8 publications

References 6 publications

Optimal Reactive Power Dispatch in ADNs using DRL and the Impact of Its Various Settings and Environmental Changes

Optimal Reactive Power Dispatch in ADNs using DRL and the Impact of Its Various Settings and Environmental Changes

Real-Time Excitation Control-Based Voltage Regulation Using DDPG Considering System Dynamic Performance

An Interpretable Planning Bot for Pancreas Stereotactic Body Radiation Therapy

Contact Info

Product

Resources

About