A Reinforcement Learning Algorithm for Market Participants in FTR Auctions

Ziogos, N.P.; Tellidou, A.C.; Gountis, V.P.; Bakirtzis, Anastasios G.

doi:10.1109/pct.2007.4538442

Cited by 7 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Q-learning is certainly one of the most studied Reinforcement Learning (RL) algorithm and has been applied with success in several domains, from relatively simple toy problems, such as Cliff-Walking (Sutton & Barto, 1998), to more complex ones, such as web-based education (Iglesias et al, 2008) and face recognition (Harandi et al, 2008). Initially proposed for single-agent environments, the simplicity and effectiveness of this algorithm has led to its application also in multiagent configurations, for example Galstyan et al (2004) and Ziogos et al (2007). In this case, however, its supporting theoretical framework and convergence guarantees are lost.…”

Section: Introductionmentioning

confidence: 99%

“…For example: Galstyan et al (2004) applies the algorithm to develop a decentralized resource allocation mechanism; Gomes and Kowalczyk (2007) study the problem of learning demand functions; and Ziogos et al (2007) investigate the development of bidding strategies. Therefore, in this paper we present a framework to model the dynamics of Multiagent Q-learning with the ǫ-greedy exploration mechanism.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic analysis of multiagent Q -learning with ε-greedy exploration

Gomes

Kowalczyk

2009

Proceedings of the 26th Annual International Conference on Machine Learning

View full text Add to dashboard Cite

The development of mechanisms to understand and model the expected behaviour of multiagent learners is becoming increasingly important as the area rapidly find application in a variety of domains. In this paper we present a framework to model the behaviour of Q-learning agents using the ǫ-greedy exploration mechanism. For this, we analyse a continuous-time version of the Q-learning update rule and study how the presence of other agents and the ǫ-greedy mechanism affect it. We then model the problem as a system of difference equations which is used to theoretically analyse the expected behaviour of the agents. The applicability of the framework is tested through experiments in typical games selected from the literature.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Dynamic analysis of multiagent Q -learning with ε-greedy exploration

Gomes

Kowalczyk

2009

Proceedings of the 26th Annual International Conference on Machine Learning

View full text Add to dashboard Cite

show abstract

“…The first FTR auction took place in 1999, in the PJM Interconnection in the U.S. In the auctions, ISOs have a goal of maximizing FTR revenues, subject to the constraints of transmission capacity and contingencies [10]. Electric suppliers calculate FTR values of the paths to bid, based on their own forecasts of future LMP prices in the interested locations.…”

Section: Financial Transmission Rightsmentioning

confidence: 99%

“…Generators submit offers, and customer loads submit bids to the ISO with hourly MWs for each hour of the next day. The ISO calculates a nodal price, or a locational marginal price (LMP) of a location, based on all the submitted offers and bids, subject to the Lagrange multipliers, or constraints of active power balance and transmission [10]. FTR auction results in ISO-NE provide the magnitude of the auctions and major FTR participants [14].…”

Section: Financial Transmission Rightsmentioning

confidence: 99%

Application of IRT Models to Selection of Bidding Paths in Financial Transmission Rights Auction: U.S. New England

Jang¹,

Jung²,

Beruvides³

2020

Energies

View full text Add to dashboard Cite

This paper explores a way to apply Item Response Theory (IRT), one of the popular statistical methodologies in measurement and psychometrics, to evaluate Financial Transmission Rights (FTR) paths in the U.S. electricity market. FTR is an energy derivative product to hedge congestion cost risks inherent in constrained transmission lines. In New England, with about 1200 pricing locations, the theoretical combinations of FTR paths amount to 1.4 million in prevailing flows alone. With capital constraints, it is imperative that FTR market participants build the capability to evaluate FTR paths to bid on. IRT provides a framework of how well tests work, and how individual items work on tests, estimating respondents’ latent abilities, and individual item parameters. IRT is utilized to analyze historical electricity data of 2019 for a daily congestion cost of eight customer load zones and one hub in the U.S., New England, for the evaluation of FTR paths. In the analysis, an item represents an FTR path, while item difficulty, item discrimination, and a latent trait variable for the path correspond to the path profitability, risk level, and daily congestion ability, respectively. This paper explores the experimental procedures by which IRT, a psychometric tool, may also be applicable in complex energy markets, providing a consistent and standardized analytical framework to address the issues of selection and prioritization among multiple opportunities. FTR path evaluation is conducted in three steps to determine bid priority paths in FTR auctions: parameter significance tests, ranking on path profitability and risk level, and weighting scores of individual rankings on the two criteria.

show abstract

“…The importance of obtaining a Q-learning algorithm with ε -greedy exploration is also justified through a large number of applications. For example, Galstyan et al [ 34 ] applied a Q-learning algorithm with ε -greedy exploration to develop a decentralised resource allocation mechanism; Gomes and Kowalczyk [ 35 ] studied the problem of learning demand functions; and Ziogos et al [ 36 ] investigated the development of bidding strategies.…”

Section: The Multi-agent Frameworkmentioning

confidence: 99%

A Multi-Agent Framework for Packet Routing in Wireless Sensor Networks

Zhang

2015

Sensors

View full text Add to dashboard Cite

Wireless sensor networks (WSNs) have been widely investigated in recent years. One of the fundamental issues in WSNs is packet routing, because in many application domains, packets have to be routed from source nodes to destination nodes as soon and as energy efficiently as possible. To address this issue, a large number of routing approaches have been proposed. Although every existing routing approach has advantages, they also have some disadvantages. In this paper, a multi-agent framework is proposed that can assist existing routing approaches to improve their routing performance. This framework enables each sensor node to build a cooperative neighbour set based on past routing experience. Such cooperative neighbours, in turn, can help the sensor to effectively relay packets in the future. This framework is independent of existing routing approaches and can be used to assist many existing routing approaches. Simulation results demonstrate the good performance of this framework in terms of four metrics: average delivery latency, successful delivery ratio, number of live nodes and total sensing coverage.

show abstract

A Reinforcement Learning Algorithm for Market Participants in FTR Auctions

Cited by 7 publications

References 14 publications

Dynamic analysis of multiagent Q -learning with ε-greedy exploration

Dynamic analysis of multiagent Q -learning with ε-greedy exploration

Application of IRT Models to Selection of Bidding Paths in Financial Transmission Rights Auction: U.S. New England

A Multi-Agent Framework for Packet Routing in Wireless Sensor Networks

Contact Info

Product

Resources

About