Design and tests of reinforcement-learning-based optimal power flow solution generator

Zhen, Hongyue; Zhai, Hefeng; Ma, Wei; Zhao, Ligang; Weng, Yixuan; Xu, Yuan; Shi, Jun; He, Xiaofeng

doi:10.1016/j.egyr.2021.11.126

Cited by 10 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhen et al [15] model the economic dispatch as 1-step Markov decision process (MDP), i.e., the solution is not generated iteratively, but in a one-shot fashion. They use the Twin-Delayed DDPG (TD3) algorithm to learn to minimize generation costs under multiple constraints.…”

Section: B Learning the Optimal Power Flowmentioning

confidence: 99%

“…Therefore, the agent has perfect knowledge of Q, if it knows the reward function. As Zhen et al [15] showed, the OPF approximation can be implemented as a 1-step environment, because the solution of one OPF is independent of the solution of the previous OPF. Exceptions are multi-step OPF problems where the optimization is done over multiple time steps, e.g., when storage systems are part of the optimization.…”

Section: Model-extended Maddpg (M-maddpg)mentioning

confidence: 99%

See 1 more Smart Citation

Towards reinforcement learning for vulnerability analysis in power-economic systems

2021

View full text Add to dashboard Cite

Future smart grids can and will be subject of systematic attacks that can result in monetary costs and reduced system stability. These attacks are not necessarily malicious, but can be economically motivated as well. Emerging flexibility markets are of interest here, because they can incite attacks if market design is flawed. The dimension and danger potential of such strategies is still unknown. Automatic analysis tools are required to systematically search for unknown strategies and their respective countermeasures. We propose deep reinforcement learning to learn attack strategies autonomously to identify underlying systemic vulnerabilities this way. As a proof-of-concept, we apply our approach to a reactive power market setting in a distribution grid. In the case study, the attacker learned to exploit the reactive power market by using controllable loads. That was done by systematically inducing constraint violations into the system and then providing countermeasures on the flexibility market to generate profit, thus finding a hitherto unknown attack strategy. As a weak-point, we identified the optimal power flow that was used for market clearing. Our general approach is applicable to detect unknown attack vectors, to analyze a specific power system regarding vulnerabilities, and to systematically evaluate potential countermeasures.

show abstract

Section: B Learning the Optimal Power Flowmentioning

confidence: 99%

Section: Model-extended Maddpg (M-maddpg)mentioning

confidence: 99%

Towards reinforcement learning for vulnerability analysis in power-economic systems

2021

View full text Add to dashboard Cite

show abstract

“…In the context of ACOPF, neural networks can either be trained by imitation (supervised learning) or by interaction with a simulator through Reinforcement Learning (RL) [7]. Recent work explores the application of deep neural networks to ACOPF [8], while others [9], [10], [11], [12] frame the ACOPF problem as a closed-loop RL problem.…”

Section: Background and Motivationsmentioning

confidence: 99%

Topology-aware reinforcement learning for tertiary voltage control

Donon,

Cubelier,

Karangelos

et al. 2024

Electric Power Systems Research

View full text Add to dashboard Cite

“…After training, the AI-based agent can adjust power flow states rapidly and is suitable for online applications. An RLbased optimal power flow solution method has been proposed in [58] using PSOPS and the twin-delayed deep deterministic policy gradient (TD3) algorithm [59]. In this paper, a TD3-based SOPF solution program is realized using Py_PSOPS.…”

Section: ) Framework Designmentioning

confidence: 99%

Exploration of Artificial-intelligence Oriented Power System Dynamic Simulators

Xiao

Chen

Wang

et al. 2023

Journal of Modern Power Systems and Clean Energy

View full text Add to dashboard Cite

With the rapid development of artificial intelligence (AI), it is foreseeable that the accuracy and efficiency of dynamic analysis for future power system will be greatly improved by the integration of dynamic simulators and AI. To explore the interaction mechanism of power system dynamic simulations and AI, a general design for AI-oriented power system dynamic simulators is proposed, which consists of a high-performance simulator with neural network supportability and flexible external and internal application programming interfaces (APIs). With the support of APIs, simulation-assisted AI and AIassisted simulation form a comprehensive interaction mechanism between power system dynamic simulations and AI. A prototype of this design is implemented and made public based on a highly efficient electromechanical simulator. Tests of this prototype are carried out in four scenarios including sample generation, AI-based stability prediction, data-driven dynamic component modeling, and AI-aided stability control, which prove the validity, flexibility, and efficiency of the design and implementation for AI-oriented power system dynamic simulators.

show abstract

Design and tests of reinforcement-learning-based optimal power flow solution generator

Cited by 10 publications

References 11 publications

Towards reinforcement learning for vulnerability analysis in power-economic systems

Towards reinforcement learning for vulnerability analysis in power-economic systems

Topology-aware reinforcement learning for tertiary voltage control

Exploration of Artificial-intelligence Oriented Power System Dynamic Simulators

Contact Info

Product

Resources

About