Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility

Charbonnier, Flora; Morstyn, Thomas; McCulloch, Malcolm

doi:10.1016/j.apenergy.2022.118825

Cited by 31 publications

(6 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is due to both the limited availability of communication and computation infrastructure at the scale of individual homes and to the privacy requirements of the residential sector [12] . Thirdly, centralised optimisation methods have limited scalability [13 , 14] . Therefore, data-driven analysis and control of the residential energy sector are of increasing interest [10 , 13] .…”

Section: Objectives and Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Home electricity data generator (HEDGE): An open-access tool for the generation of electric vehicle, residential demand, and PV generation profiles

Charbonnier,

Morstyn,

McCulloch

2024

MethodsX

View full text Add to dashboard Cite

Section: Objectives and Motivationmentioning

confidence: 99%

“…Thirdly, centralised optimisation methods have limited scalability [13 , 14] . Therefore, data-driven analysis and control of the residential energy sector are of increasing interest [10 , 13] . Fig.…”

Section: Objectives and Motivationmentioning

confidence: 99%

Home electricity data generator (HEDGE): An open-access tool for the generation of electric vehicle, residential demand, and PV generation profiles

Charbonnier,

Morstyn,

McCulloch

2024

MethodsX

View full text Add to dashboard Cite

“…The agents can be implemented by functions, methods, processes, algorithms, etc. Multiagent reinforcement learning has been widely applied in fields such as robot cooperation, human-machine chess, autonomous driving, distributed control resource management, collaborative decision support systems, autonomous combat systems, and data mining [87][88][89].…”

Section: Multiagent Reinforcement Learningmentioning

confidence: 99%

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Wu,

Zhang,

Miao

et al. 2024

Electronics

View full text Add to dashboard Cite

Extensive research has been carried out on reinforcement learning methods. The core idea of reinforcement learning is to learn methods by means of trial and error, and it has been successfully applied to robotics, autonomous driving, gaming, healthcare, resource management, and other fields. However, when building reinforcement learning solutions at the edge, not only are there the challenges of data-hungry and insufficient computational resources but also there is the difficulty of a single reinforcement learning method to meet the requirements of the model in terms of efficiency, generalization, robustness, and so on. These solutions rely on expert knowledge for the design of edge-side integrated reinforcement learning methods, and they lack high-level system architecture design to support their wider generalization and application. Therefore, in this paper, instead of surveying reinforcement learning systems, we survey the most commonly used options for each part of the architecture from the point of view of integrated application. We present the characteristics of traditional reinforcement learning in several aspects and design a corresponding integration framework based on them. In this process, we show a complete primer on the design of reinforcement learning architectures while also demonstrating the flexibility of the various parts of the architecture to be adapted to the characteristics of different edge tasks. Overall, reinforcement learning has become an important tool in intelligent decision making, but it still faces many challenges in the practical application in edge computing. The aim of this paper is to provide researchers and practitioners with a new, integrated perspective to better understand and apply reinforcement learning in edge decision-making tasks.

show abstract

Section: Energymentioning

confidence: 99%

“…This work formulates the problem of POMG and MESS agents interacting with the environment and making independent decisions for simultaneous routing and scheduling based on local information. Charbonnier et al (2022) presented a tabular MA-QL approach for energy coordination management. Agents are proactive consumers, and local observations are modeled in a Dec-POMDP environment; they make individual decisions to find a trade-off between local, grid, and social objectives.…”

Section: Applicationsmentioning

confidence: 99%

A review of the applications of multi-agent reinforcement learning in smart factories

Bahrpeyma

Reichelt

2022

Front. Robot. AI

View full text Add to dashboard Cite

The smart factory is at the heart of Industry 4.0 and is the new paradigm for establishing advanced manufacturing systems and realizing modern manufacturing objectives such as mass customization, automation, efficiency, and self-organization all at once. Such manufacturing systems, however, are characterized by dynamic and complex environments where a large number of decisions should be made for smart components such as production machines and the material handling system in a real-time and optimal manner. AI offers key intelligent control approaches in order to realize efficiency, agility, and automation all at once. One of the most challenging problems faced in this regard is uncertainty, meaning that due to the dynamic nature of the smart manufacturing environments, sudden seen or unseen events occur that should be handled in real-time. Due to the complexity and high-dimensionality of smart factories, it is not possible to predict all the possible events or prepare appropriate scenarios to respond. Reinforcement learning is an AI technique that provides the intelligent control processes needed to deal with such uncertainties. Due to the distributed nature of smart factories and the presence of multiple decision-making components, multi-agent reinforcement learning (MARL) should be incorporated instead of single-agent reinforcement learning (SARL), which, due to the complexities involved in the development process, has attracted less attention. In this research, we will review the literature on the applications of MARL to tasks within a smart factory and then demonstrate a mapping connecting smart factory attributes to the equivalent MARL features, based on which we suggest MARL to be one of the most effective approaches for implementing the control mechanism for smart factories.

show abstract

Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility

Cited by 31 publications

References 52 publications

Home electricity data generator (HEDGE): An open-access tool for the generation of electric vehicle, residential demand, and PV generation profiles

Home electricity data generator (HEDGE): An open-access tool for the generation of electric vehicle, residential demand, and PV generation profiles

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

A review of the applications of multi-agent reinforcement learning in smart factories

Contact Info

Product

Resources

About