Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach

Qiu, Dawei; Ye, Yujian; Papadaskalopoulos, Dimitrios; Štrbac, Goran

doi:10.1016/j.apenergy.2021.116940

Cited by 107 publications

(43 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If RL is used to minimize gasoline consumption of a PHEV, the gasoline tank and the engine should be modelled in the environment at a suitable level of abstraction [13]. The availability of V2G (vehicle-to-grid) needs to be considered when modelling the possibility to sell energy from the vehicle batteries to the grid [14], but details such as grid inverters may be abstracted away at the discretion of the authors [15]. For a wireless EV charging system, the EV characteristics and the traffic environment need to be considered [16].…”

Section: General Conceptual Overview For Reinforcement Learning Agent...mentioning

confidence: 99%

“…An aggregator can trade the capacity of the batteries and other flexible energy resources on utility markets [22], [145], [130]. Alternatively, a local market can be established to avoiding buying and selling from the grid [143], [158], [118], [14]. Huang et al [93] take battery management as one criterion in a multi-objective optimization that aims to reduce the latencies and dropped packets for the IoT computation tasks.…”

Section: Buildingmentioning

confidence: 99%

See 1 more Smart Citation

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

2022

View full text Add to dashboard Cite

The transition to renewable production and smart grids is driving a massive investment to battery storages, and reinforcement learning (RL) has recently emerged as a potentially disruptive technology for their control and optimization of battery storage systems. A surge of papers has appeared in the last two years applying reinforcement learning to the optimization of battery storages in buildings, energy communities, energy harvesting Internet of Things networks, renewable generation, microgrids, electric vehicles and plug-in hybrid electric vehicles. This article reviews these applications through 4 different perspectives. Firstly, the type of optimization problem is analyzed; the literature can be divided to approaches that optimize either financial targets or energy efficiency. Secondly, the approaches for handling user comfort are analyzed for applications that may impact a human user. Thirdly, this paper discusses the approach to model and reduce battery degradation. Fourthly, the articles are categorized by application context and applications likely to attract a high amount of research are identified. The paper concludes with a list of unresolved challenges.

show abstract

Section: General Conceptual Overview For Reinforcement Learning Agent...mentioning

confidence: 99%

Section: Buildingmentioning

confidence: 99%

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

2022

View full text Add to dashboard Cite

show abstract

“…Application Objective Building Type Algorithm [141], [142] Other/Mixed Cost Residential DQN [143] Cost & Load Balance [94] EV, ES, and RG Cost [144] Other [145] Cost & Comfort [146] HVAC, Fans, WH Cost [147] Other/Mixed Commercial [148] Cost & Comfort [149], [150] HVAC, Fans, WH Mixed/NA [151], [152] Other/Mixed Cost [153], [154] P2P Trading Other Mixed/NA [163] EV, ES, and RG [164] Other/Mixed Cost [165] Cost & Comfort Residential TRPO [51], [168], [169], [170] Other/Mixed [171], [172] Cost & Load Balance [173] Cost [174] EV, ES, and RG [175] Other/Mixed Cost & Comfort Academic [176] Other [177], [178] EV, ES, and RG Commercial [179], [180], [181] HVAC, Fans, WH Cost & Comfort Mixed/NA [182], [183], [184] EV, ES, and RG Other [185], [186] Other/Mixed Cost & Load Balance Residential SAC [187], [188] HVAC, Fans, WH Cost Commercial [189], [190],…”

Section: Referencementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

The twin capabilities of learning from experience and learning at higher levels of abstraction, set reinforcement learning apart from other areas of machine learning and (within the broader context) all of artificial intelligence. It allows algorithmic agents to replace human beings in the real world, including in homes and buildings, in application domains that had hitherto been considered to be beyond today’s capabilities. This goal, specifically aimed at home energy automation that forms the backdrop of this article, which surveys the use of deep reinforcement learning in various HEMS applications. The article provides an overview of generic reinforcement learning. This is followed with discussions on the state-of-the-art methods for value based, policy gradient, and actor-critic methods in deep reinforcement learning. In order to make published literature in reinforcement learning more accessible to HEMS researchers, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using the same terminology as the machine learning community. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. The survey suggests that this research is still in its infancy.

show abstract

“…Building Type Algorithm [152,153] Other/Mixed Cost Residential DQN [154] Cost and Load Balance [105] EV, ES, and RG Cost [155] Other [156] Cost and Comfort [157] HVAC, Fans, WH Cost [158] Other/Mixed Commercial [159] Cost and Comfort [160,161] EV, ES, and RG [175] Other/Mixed Cost [176] Cost and Comfort Residential TRPO Other/Mixed [182,183] Cost and Load Balance [184] Cost [185] EV, ES, and RG [186] Other/Mixed Cost and Comfort Academic [187] Other [188,189] EV, ES, and RG Commercial [190][191][192] HVAC, Fans, WH Cost and Comfort Mixed/NA [193][194][195] EV, ES, and RG Other [196,197] Other/Mixed Cost and Load Balance Residential SAC [198,199] HVAC, Fans, WH Cost Commercial [103,[200][201][202] Cost and Comfort [203] Other/Mixed [204] Academic [205][206][207] HVAC, Fans, WH Cost and Load Balance Mixed/NA [208][209][210]<...…”

Section: Reference Application Objectivementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

The steep rise in reinforcement learning (RL) in various applications in energy as well as the penetration of home automation in recent years are the motivation for this article. It surveys the use of RL in various home energy management system (HEMS) applications. There is a focus on deep neural network (DNN) models in RL. The article provides an overview of reinforcement learning. This is followed with discussions on state-of-the-art methods for value, policy, and actor–critic methods in deep reinforcement learning (DRL). In order to make the published literature in reinforcement learning more accessible to the HEMS community, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using standard machine learning terminology. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. It suggests that research in this direction is still in its infancy. Lastly, the article proposes four performance metrics to evaluate RL methods.

show abstract

Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach

Cited by 107 publications

References 37 publications

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

Reinforcement Learning: Theory and Applications in HEMS

Reinforcement Learning: Theory and Applications in HEMS

Contact Info

Product

Resources

About