Reinforcement learning in local energy markets

Bose, Samrat; Kremers, Enrique; Mengelkamp, Esther; Eberbach, Jan; Weinhardt, Christof

doi:10.1186/s42162-021-00141-z

Cited by 13 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By this, the market agents consider the results of the historical market rounds as well as the success of the bids, which is incorporated into the probability distribution for future bidding [13]. [14,15] have shown that the Roth-Erev learning algorithm enables the households to achieve promising economic benefits in energy trading.…”

Section: Local Energy Market Modelmentioning

confidence: 99%

Evaluation of Energy Sharing on a Local Energy Market Through Comparison of Energy Management Techniques

Schölzel,

Henn,

Streblow

et al. 2023

36th International Conference on Efficiency, Cost, Optimization, Simulation and Environmental Impact of Energy Systems (ECOS 20

View full text Add to dashboard Cite

Local energy markets (LEMs) are a promising way to solve the challenges of the increasing extension of decentralized energy systems and to promote the further integration of renewable energy sources. LEMs enable costumers with distributed energy resources to trade and share their electrical energy with each other. In the existing literature, the research focus is mostly on the development and evaluation of specific elements of LEMs, such as bidding strategies or market designs. The paper contributes a comprehensive evaluation of a LEM and the quantification of its benefit regarding the market-based device operation. For the evaluation in terms of financial outcome and local energy exchange, a centralized and a decentralized operation optimization serve as upper and lower references. In centralized optimization, the system boundary comprises the entire neighbourhood. In decentralized optimization, each building is balanced separately. For the LEM, we introduce a distributed market design with the involvement of an auctioneer. We focus there on the implementation of learning bidding strategies and a double-sided auction with non-iterative market clearing rules. For all three energy management techniques, the operating schedules of the devices are determined using mixed-integer linear programming. In several case studies we investigate different neighbourhoods in order to evaluate the influence of different technologies and their penetrations as well as the impact of the building stock in terms of building type and construction year. We evaluate the market outcome with multiple key performance indicators (KPIs) such as the supply-and demand-cover-factor, the total operation costs and the peak load. The results show that total energy costs can be reduced by up to 6.4 %. For the energy exchange, it is shown that the electricity surplus is up to 72 % and the electricity demand of the QUartier decreases by up to 6.8 % compared to the decentralized optimization and increases by up to 14.3 % compared to the centralized optimization. Further, we noted up to 46.2 % higher peak loads.

show abstract

Section: Local Energy Market Modelmentioning

confidence: 99%

Evaluation of Energy Sharing on a Local Energy Market Through Comparison of Energy Management Techniques

Schölzel,

Henn,

Streblow

et al. 2023

36th International Conference on Efficiency, Cost, Optimization, Simulation and Environmental Impact of Energy Systems (ECOS 20

View full text Add to dashboard Cite

show abstract

“…Bose et al [6] simulate a local energy market as a multi-agent simulation of 100 households. Through the use of the Roth-Erev reinforcement learning algorithm to control trading, and demand response of electricity.…”

Section: Literature Review 41 Reinforcement Learningmentioning

confidence: 99%

“…Market Type Application Algorithm Used 2021 Bose S. [6] Local energy market Peer to peer trading Roth-Erev 2021 Naseri N. [55] Local energy market Tariff design SARIMAX, MDP 2021 Tang C. [78] International/National Tariff design Novel WoLF-PHC 2021 Liu D. [42] International/National Bidding strategies, Peerto-Peer MADDPG 2021 Deng C. [12] International/National Demand response DDPG 2021 Viehmann J. [82] International/National Market investigation Q-Learning 2020 Tomin N. [80] Microgrid Electricity grid control Q-Learning 2020 Liang Y.…”

Section: Year First Authormentioning

confidence: 99%

Machine learning applications for electricity market agent-based models: A systematic literature review

Kell¹,

McGough²,

Forshaw³

2022

Preprint

View full text Add to dashboard Cite

The electricity market has a vital role to play in the decarbonisation of the energy system. However, the electricity market is made up of many different variables and data inputs. These variables and data inputs behave in sometimes unpredictable ways which can not be predicted a-priori. It has therefore been suggested that agentbased simulations are used to better understand the dynamics of the electricity market. Agent-based models provide the opportunity to integrate machine learning and artificial intelligence to add intelligence, make better forecasts and control the power market in better and more efficient ways. In this systematic literature review, we review 55 papers published between 2016 and 2021 which focus on machine learning applied to agent-based electricity market models. We find that research clusters around popular topics, such as bidding strategies. However, there exists a long-tail of different research applications that could benefit from the high intensity research from the more investigated applications. CCS CONCEPTS• Computing methodologies → Modeling and simulation; Machine learning; Artificial intelligence.

show abstract

“…The proposed trading mechanism in [20] sought to maximize the trading profit of prosumers while also considering the fair trading profit of prosumers. Bose et al [21] proposed a RL-based trading model that gives the option to prosumers to choose between different trading strategies with different gains and penalties. Kim et al [22] proposed a RL-based trading model with a new trading evaluation criterion that considers the main factors in LEM.…”

Section: P2p Energy Trade Systemmentioning

confidence: 99%

“…Herein, the implementation issues, including grid network constraints or trading platforms, are not discussed. We focused on designing a trade management algorithm for prosumers in the P2P energy trade system with assumptions similar to those in previous studies [17][18][19][20][21][22], but we employed the CES that were not considered in [17][18][19][20][21][22].…”

Section: P2p Energy Trade Systemmentioning

confidence: 99%

Reinforcement Learning Based Peer-to-Peer Energy Trade Management Using Community Energy Storage in Local Energy Market

Zang

Kim

2021

Energies

View full text Add to dashboard Cite

Many studies have proposed a peer-to-peer energy market where the prosumers’ actions, including energy consumption, charge and discharge schedule of energy storage systems, and transactions in local energy markets, are controlled by a central operator. In this paper, prosumers’ actions are not controlled by an operator, and the prosumers freely participate in the local energy market to trade energy with other prosumers. We designed and modeled a local energy market with a management algorithm that uses community energy storage for prosumers who competitively participate in trade in the real-time energy market. We propose an energy-trade management algorithm that manages the trades of prosumers in two phases based on bids and offers submitted by prosumers. The first phase is to manage the trade of prosumers who have submitted fair prices to trade with other prosumers in the real-time energy market. The second phase is managing the trade of prosumers that could not trade in the first phase. Community energy storage is employed in the second phase and controlled by a reinforcement learning-based trading algorithm to decide whether to buy, sell, or do nothing with the prosumers. The action of buying and selling means charging and discharging the community energy storage, respectively. Numerical results show that the proposed trading algorithm gains a near-maximum profit. Besides, we verified that community energy storage yields more profit than the battery wear-out cost.

show abstract

Reinforcement learning in local energy markets

Cited by 13 publications

References 42 publications

Evaluation of Energy Sharing on a Local Energy Market Through Comparison of Energy Management Techniques

Evaluation of Energy Sharing on a Local Energy Market Through Comparison of Energy Management Techniques

Machine learning applications for electricity market agent-based models: A systematic literature review

Reinforcement Learning Based Peer-to-Peer Energy Trade Management Using Community Energy Storage in Local Energy Market

Contact Info

Product

Resources

About