Application of Reinforcement Learning to Electrical Power System Closed-Loop Emergency Control

Druet, Christophe; Ernst, Damien; Wehenkel, Louis

doi:10.1007/3-540-45372-5_9

Cited by 8 publications

(6 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The earliest research reports date back to 1999 and 2000 [19,22,23]. In [19] the use of RL in electric power system closed-loop emergency control, were investigated.…”

Section: Results Under Cyclic Power Demand Variationsmentioning

confidence: 99%

“…RELATED WORK The application of RL algorithms to power system stability control is still in its infancy. Considerable research efforts have been done at the University of Liège [16,[19][20][21] and this paper is a result of those efforts.…”

Section: Results Under Cyclic Power Demand Variationsmentioning

confidence: 99%

“…In [19] the use of RL in electric power system closed-loop emergency control, were investigated. A RL algorithm was used to determine a discrete switching control law to trip generators so as to avoid loss of synchronism.…”

Section: Results Under Cyclic Power Demand Variationsmentioning

confidence: 99%

See 2 more Smart Citations

Power Systems Stability Control: Reinforcement Learning Framework

Ernst

Glavić

Wehenkel

2004

IEEE Trans. Power Syst.

Self Cite

185

View full text Add to dashboard Cite

Abstract-In this paper we explore how a computational approach to learning from interactions, called Reinforcement Learning (RL), can be applied to control power systems. We describe some challenges in power system control and discuss how some of those challenges could be met by using these RL methods. The difficulties associated with their application to control power systems are described and discussed as well as strategies that can be adopted to overcome them. Two reinforcement learning modes are considered : the on-line mode in which the interaction occurs with the real power system and the off-line mode in which the interaction occurs with a simulation model of the real power system. We present two case studies made on a 4-machine power system model. The first one concerns the design by means of RL algorithms used in off-line mode of a dynamic brake controller. The second concerns RL methods used in online mode when applied to control a Thyristor Controlled Series Capacitor (TCSC) aimed to damp power system oscillations.

show abstract

“…The earliest research reports date back to 1999 and 2000 [19,22,23]. In [19] the use of RL in electric power system closed-loop emergency control, were investigated.…”

Section: Results Under Cyclic Power Demand Variationsmentioning

confidence: 99%

Section: Results Under Cyclic Power Demand Variationsmentioning

confidence: 99%

See 1 more Smart Citation

Power Systems Stability Control: Reinforcement Learning Framework

Ernst

Glavić

Wehenkel

2004

IEEE Trans. Power Syst.

Self Cite

185

View full text Add to dashboard Cite

show abstract

“…While the learning stage refers to the usual RL implementation, the execution stage deploy the knowledge acquired from the learning stage for decision making. As TSA crisis can be considered as a widearea control systems' (WAC) crisis, Druet et al [112] investigated the deployment of RL using Monte Carlo control to define the switching control law for tripping generators in order to avoid loss of synchronism. However, due to scalability challenges, traditional RL algorithms struggle especially with regards to large scale power system.…”

Section: ) Review Of Rl and Drl Approaches To Tsamentioning

confidence: 99%

A Review of Machine Learning Approaches to Power System Security and Stability

2020

View full text Add to dashboard Cite

Increasing use of renewable energy sources, liberalized energy markets and most importantly, the integrations of various monitoring, measuring and communication infrastructures into modern power system network offer the opportunity to build a resilient and efficient grid. However, it also brings about various threats of instabilities and security concerns in form of cyberattack, voltage instability, power quality (PQ) disturbance among others to the complex network. The need for efficient methodologies for quicker identification and detection of these problems have always been a priority to energy stakeholders over the years. In recent times, machine learning techniques (MLTs) have proven to be effective in numerous applications including power system studies. In the literature, various MLTs such as artificial neural networks (ANN), Decision Tree (DT), support vector machines (SVM) have been proposed, resulting in effective decision making and control actions in the secured and stable operations of the power system. Given this growing trend, this paper presents a comprehensive review on the most recent studies whereby MLTs were developed for power system security and stability especially in cyberattack detections, PQ disturbance studies and dynamic security assessment studies. The aim is to highlight the methodologies, achievements and more importantly the limitations in the classifier(s) design, dataset and test systems employed in the reviewed publications. A brief review of reinforcement learning (RL) and deep reinforcement learning (DRL) approaches to transient stability assessment is also presented. Finally, we highlighted some challenges and directions for future studies.

show abstract

“…RL methods can solve sequential decisionmaking problems in real time [10]. The last two decades have seen increasing efforts to apply conventional RL methods, such as Q-learning and fitted Q-iteration [10], in various decisionmaking and control problems in power systems; these range from demand response [11], energy management, and automatic generation control to transient stability and emergency control [9], [12], [13]. Due to scalability issues, applications of conventional RL methods are mainly focusing on problems with low-dimensional state and action spaces.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Power System Emergency Control Using Deep Reinforcement Learning

Huang

Hao

et al. 2020

IEEE Trans. Smart Grid

288

157

View full text Add to dashboard Cite

Power system emergency control is generally regarded as the last safety net for grid security and resiliency. Existing emergency control schemes are usually designed off-line based on either the conceived "worst" case scenario or a few typical operation scenarios. These schemes are facing significant adaptiveness and robustness issues as increasing uncertainties and variations occur in modern electrical grids. To address these challenges, this paper developed novel adaptive emergency control schemes using deep reinforcement learning (DRL) by leveraging the high-dimensional feature extraction and non-linear generalization capabilities of DRL for complex power systems.Furthermore, an open-source platform named Reinforcement Learning for Grid Control (RLGC) has been designed for the first time to assist the development and benchmarking of DRL algorithms for power system control. Details of the platform and DRL-based emergency control schemes for generator dynamic braking and under-voltage load shedding are presented. Robustness of the developed DRL method to different simulation scenarios, model parameter uncertainty and noise in the observations is investigated. Extensive case studies performed in both the two-area, four-machine system and the IEEE 39-bus system have demonstrated excellent performance and robustness of the proposed schemes.

show abstract

Application of Reinforcement Learning to Electrical Power System Closed-Loop Emergency Control

Cited by 8 publications

References 4 publications

Power Systems Stability Control: Reinforcement Learning Framework

Power Systems Stability Control: Reinforcement Learning Framework

A Review of Machine Learning Approaches to Power System Security and Stability

Adaptive Power System Emergency Control Using Deep Reinforcement Learning

Contact Info

Product

Resources

About