Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning

Zhu, Ziqing; Chan, Ka Wing; Bu, Siqi; Or, Siu Wing; Gao, Xiang; Xia, Shiwei

doi:10.1109/tpwrs.2021.3099693

Cited by 19 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They use data from a single day for training and the market is cleared with a constrained DC OPF. In follow-up work, Zhu et al [12] use the same algorithm to identify load demand, congestions, and price caps as key factors that affect the converged bidding strategies.…”

Section: A Multi-agent Bidding In Energy Marketsmentioning

confidence: 99%

Towards reinforcement learning for vulnerability analysis in power-economic systems

2021

View full text Add to dashboard Cite

Future smart grids can and will be subject of systematic attacks that can result in monetary costs and reduced system stability. These attacks are not necessarily malicious, but can be economically motivated as well. Emerging flexibility markets are of interest here, because they can incite attacks if market design is flawed. The dimension and danger potential of such strategies is still unknown. Automatic analysis tools are required to systematically search for unknown strategies and their respective countermeasures. We propose deep reinforcement learning to learn attack strategies autonomously to identify underlying systemic vulnerabilities this way. As a proof-of-concept, we apply our approach to a reactive power market setting in a distribution grid. In the case study, the attacker learned to exploit the reactive power market by using controllable loads. That was done by systematically inducing constraint violations into the system and then providing countermeasures on the flexibility market to generate profit, thus finding a hitherto unknown attack strategy. As a weak-point, we identified the optimal power flow that was used for market clearing. Our general approach is applicable to detect unknown attack vectors, to analyze a specific power system regarding vulnerabilities, and to systematically evaluate potential countermeasures.

show abstract

Section: A Multi-agent Bidding In Energy Marketsmentioning

confidence: 99%

Towards reinforcement learning for vulnerability analysis in power-economic systems

2021

View full text Add to dashboard Cite

show abstract

“…Reward of this research is the trading cost or revenue of the participant, and action is its bidding decision. Typical reward and action can be expressed by equations ( 16) and ( 17) [72][73][74][75][76].…”

Section: Electricity Marketmentioning

confidence: 99%

Application of reinforcement learning in planning and operation of new power system towards carbon peaking and neutrality

Sun¹,

Wang²,

Huang³

et al. 2023

Prog. Energy

View full text Add to dashboard Cite

To mitigate global climate change and ensure a sustainable energy future, China has launched a new energy policy of achieving carbon peaking by 2030 and carbon neutrality by 2060, which sets an ambitious goal of building New Power System (NPS) with high penetration of renewable energy. However, the strong uncertainty, nonlinearity, and intermittency of renewable generation and their power electronics-based control devices are imposing grand challenges for secure and economic planning and operation of the NPS. The performance of traditional methods and tools becomes rather limited under such phenomena. Together with high-fidelity modeling and high-performance simulation techniques, the fast development of artificial intelligence (AI) technology, especially reinforcement learning (RL), provides a promising way of tackling these critical issues. This paper first provides a comprehensive overview of RL methods that interact with high-fidelity grid simulators to train effective agents for intelligent, model-free decision-making. Secondly, three important applications of RL are reviewed, including device-level control, system-level optimized control, and demand side management, with detailed modeling and procedures of solution explained. Finally, this paper discusses future research efforts for achieving the goals of full absorption of renewable energy, optimized allocation of large-scale energy resources, reliable supply of electricity, and secure and economic operation of the power grid.

show abstract

Demand response management of smart grid based on Stackelberg-evolutionary joint game

Dong

2023

Sci. China Inf. Sci.

View full text Add to dashboard Cite

Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning

Cited by 19 publications

References 9 publications

Towards reinforcement learning for vulnerability analysis in power-economic systems

Towards reinforcement learning for vulnerability analysis in power-economic systems

Application of reinforcement learning in planning and operation of new power system towards carbon peaking and neutrality

Demand response management of smart grid based on Stackelberg-evolutionary joint game

Contact Info

Product

Resources

About