A Reinforcement Learning approach for the continuous electricity market of Germany: Trading from the perspective of a wind park operator

Lehna, Malte; Hoppmann, Björn; Heinrich, René; Scholz, Christoph

doi:10.1016/j.egyai.2022.100139

Cited by 13 publications

(4 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our main conclusion is that DDPG can find Nash equilibria in our benchmark scenario. Therefore, DDPG is a useful algorithm in the price competition game: first, we want to highlight that algorithmic-trading has become a commodity in electricity market trading (see, e.g., Lehna et al, 2022 , and literature references therein), and we believe that DDPG could be a viable candidate to be deployed in that domain. Therefore some of the actions performed on real markets may follow DDPG’s logic in the future.…”

Section: How To Model Strategic Interactions? From Game-theoretic Equ...mentioning

confidence: 99%

Computational Performance of Deep Reinforcement Learning to Find Nash Equilibria

et al. 2023

View full text Add to dashboard Cite

We test the performance of deep deterministic policy gradient—a deep reinforcement learning algorithm, able to handle continuous state and action spaces—to find Nash equilibria in a setting where firms compete in offer prices through a uniform price auction. These algorithms are typically considered “model-free” although a large set of parameters is utilized by the algorithm. These parameters may include learning rates, memory buffers, state space dimensioning, normalizations, or noise decay rates, and the purpose of this work is to systematically test the effect of these parameter configurations on convergence to the analytically derived Bertrand equilibrium. We find parameter choices that can reach convergence rates of up to 99%. We show that the algorithm also converges in more complex settings with multiple players and different cost structures. Its reliable convergence may make the method a useful tool to studying strategic behavior of firms even in more complex settings.

show abstract

Section: How To Model Strategic Interactions? From Game-theoretic Equ...mentioning

confidence: 99%

Computational Performance of Deep Reinforcement Learning to Find Nash Equilibria

et al. 2023

View full text Add to dashboard Cite

show abstract

“…RL-based models account for most of the research. The Q-learning algorithm is a classic and interpretable RL algorithm that is widely used in several models to simulate the behavior of various market participants [137,140,141,146,152,153] . However, the Q-learning algorithm is weak in dealing with the continuous bidding action space.…”

Section: Multi-agent-based Models and Solving Methodsmentioning

confidence: 99%

The competition and equilibrium in power markets under decarbonization and decentralization

Chen

Fang

Guo

et al. 2022

iEnergy

View full text Add to dashboard Cite

Equilibrium analysis has been widely studied as an effective tool to model gaming interactions and predict market results. However, as competition modes are fundamentally changed by the decarbonization and decentralization of power systems, analysis techniques must evolve. This article comprehensively reviews recent developments in modelling methods, practical settings and solution techniques in equilibrium analysis. Firstly, we review equilibrium in the evolving wholesale power markets which feature new entrants, novel trading products and multi-stage clearing. Secondly, the competition modes in the emerging distribution market and distributed resource aggregation are reviewed, and we compare peer-to-peer clearing, cooperative games and Stackelberg games. Furthermore, we summarize the methods to treat various information acquisition degrees, risk preferences and rationalities of market participants. To deal with increasingly complex market settings, this review also covers refined analytical techniques and agent-based models used to compute the equilibrium. Finally, based on this review, this paper summarizes key issues in the gaming and equilibrium analysis in power markets under decarbonization and decentralization.

show abstract

“…RL algorithms can learn to analyse historical load data and external factors such as weather patterns and events to predict the future load demand accurately. Main RL applications in load forecasting are Short-term load forecasting [17]- [18], Renewable energy forecasting [19] and Energy market forecasting [20].…”

Section: Reinforcement Learningmentioning

confidence: 99%

Reinforcement Learning in Power System Control and Optimization

Bernadić¹,

Kujundžić²,

Primorac

2023

B&H Electrical Engineering

View full text Add to dashboard Cite

Reinforcement learning (RL) is area of Machine Learning (ML) and part of wide-range portfolio of the Artificial intelligence (AI) methods. Besides the explanations of the concepts and principles of RL, in the paper are presented practical RL models for control and optimizing operation of power system – controlling tap-changers for maintain voltage levels and model for techno-economical optimizing operation of energy storages of households in microgrid. Trained RL agent in the practical example synchronizes operation of tap-changers to maintain satisfactory voltage level for the consumers, even in the network with distributed generation. Energy storages are in wide use in households, especially in the combination with PV. In the second example, microgrid’s energy management system (EMS) RL agent after learning process act in the simulated environment with variable electrical energy prices, variable load profiles and efficiency of PV modules of households to maximize profit for the houseowners in the microgrid. Agent controls charging and discharging of energy storages and obtain maximal benefit in randomly determined conditions of microgrid operation and different tariff situations. Models are implemented in the Python programming environment Python with specialized power system simulation software (Pandapower) and RL libraries (RLib, OpenAI).

show abstract

A Reinforcement Learning approach for the continuous electricity market of Germany: Trading from the perspective of a wind park operator

Cited by 13 publications

References 35 publications

Computational Performance of Deep Reinforcement Learning to Find Nash Equilibria

Computational Performance of Deep Reinforcement Learning to Find Nash Equilibria

The competition and equilibrium in power markets under decarbonization and decentralization

Reinforcement Learning in Power System Control and Optimization

Contact Info

Product

Resources

About