A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding

Ernst, Damien; Théate, Thibaut; Bolland, Adrien; Huynen, Alexandre; Buchwald, Martin; Wynants, Christelle; Cornélusse, Bertrand

doi:10.48550/arxiv.2004.05940

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this work does not explicitly model the trading process in the ID market, and only considers a limited order book (a compilation of available bids and asks for trading). More thorough modeling of the CID market is provided by [15], where a deep reinforcement learning framework for the participation of energy storage in the CID market is proposed. In the latter, historical limit order book data were used to train the trading agent.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Analyzing Trade in Continuous Intra-Day Electricity Market: An Agent-Based Modeling Approach

et al. 2021

View full text Add to dashboard Cite

In recent years, the vast penetration of renewable energy sources has introduced a large degree of uncertainty into the power system, thus leading to increased trading activity in the continuous intra-day electricity market. In this paper, we propose an agent-based modeling framework to analyze the behavior and the interactions between renewable energy sources, consumers and thermal power plants in the European Continuous Intra-day (CID) market. Additionally, we propose a novel adaptive trading strategy that can be used by the agents that participate in CID market. The agents learn how to adapt their behavior according to the arrival of new information and how to react to changing market conditions by updating their willingness to trade. A comparative analysis was performed to study the behavior of agents when they adopt the proposed strategy as opposed to other benchmark strategies. The effects of unexpected outages and information asymmetry on the market evolution and the market liquidity were also investigated.

show abstract

Section: Related Workmentioning

confidence: 99%

“…This section describes the organization and the main principles that define the exchanges in the proposed CID market mechanisms. These principles are inspired by the work of Boukas et al; for more details, we refer the reader to [15].…”

Section: Principles Of Continuous Intra-day Marketsmentioning

confidence: 99%

Analyzing Trade in Continuous Intra-Day Electricity Market: An Agent-Based Modeling Approach

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The work of (Xiong et al, 2018) provides a framework for Deterministic Policy Gradients in optimizing daily trades using additional financial metrics and cumulative returns. (Boukas et al, 2020) extends the RL trading strategy towards intraday execution comprising of asynchronous distributed updates in conjunction with backtesting methods. The resulting method is found to be sample efficient.…”

Section: Trade Execution Using Rlmentioning

confidence: 99%

TradeR: Practical Deep Hierarchical Reinforcement Learning for Trade Execution

Suri,

Shi,

Plataniotis

et al. 2021

Preprint

View full text Add to dashboard Cite

Advances in Reinforcement Learning (RL) span a wide variety of applications which motivate development in this area. While application tasks serve as suitable benchmarks for real world problems, RL is seldomly used in practical scenarios consisting of abrupt dynamics. This allows one to rethink the problem setup in light of practical challenges. We present Trade Execution using Reinforcement Learning (TradeR) which aims to address two such practical challenges of catastrophy and surprise minimization by formulating trading as a real-world hierarchical RL problem. Through this lens, TradeR makes use of hierarchical RL to execute trade bids on highfrequency real market experiences comprising of abrupt price variations during the 2019 fiscal year COVID19 stock market crash. The framework utilizes an energy-based scheme in conjunction with surprise value function for estimating and minimizing surprise. In a large-scale study of 35 stock symbols from the S&P500 index, TradeR demonstrates robustness to abrupt price changes and catastrophic losses while maintaining profitable outcomes. We hope that our work serves as a motivating example for application of RL to practical problems.

show abstract

“…The state of charge of the battery is updated using a linear water tank model [Boukas et al, 2020]. With this tank model, the value of SoC t+1 at time t + 1, if there were no limits on it, would be equal to A t+1 defined as follows:…”

mentioning

confidence: 99%

Jointly Learning Environments and Control Policies with Projected Stochastic Gradient Ascent

Bolland¹,

Berger²,

Ernst³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this work, we generalize the direct policy search algorithms to an algorithm we call Direct Environment Search with (projected stochastic) Gradient Ascent (DESGA). The latter can be used to jointly learn a reinforcement learning (RL) environment and a policy with maximal expected return over a joint hypothesis space of environments and policies. We illustrate the performance of DESGA on two benchmarks. First, we consider a parametrized space of Mass-Spring-Damper (MSD) environments. Then, we use our algorithm for optimizing the size of the components and the operation of a small-scale and autonomous energy system, i.e. a solar off-grid microgrid, composed of photovoltaic panels, batteries, etc. The results highlight the excellent performances of the DESGA algorithm.Preprint. Under review.

show abstract

A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding

Cited by 3 publications

References 0 publications

Analyzing Trade in Continuous Intra-Day Electricity Market: An Agent-Based Modeling Approach

Analyzing Trade in Continuous Intra-Day Electricity Market: An Agent-Based Modeling Approach

TradeR: Practical Deep Hierarchical Reinforcement Learning for Trade Execution

Jointly Learning Environments and Control Policies with Projected Stochastic Gradient Ascent

Contact Info

Product

Resources

About