Sharing of Energy Among Cooperative Households Using Distributed Multi-Agent Reinforcement Learning

Ebell, Niklas; Gütlein, Moritz; Pruckner, Marco

doi:10.1109/isgteurope.2019.8905520

Cited by 11 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This kind of implementation is also typical for cooperative network agents in RL. Ebell et al [126] applied the MARL to develop a decentralized control strategy for household energy cells where distributed Q-learning strategies of the agents under partial and full observability of the environment are both investigated. As for resources management, Xiao [127] proposed a distributed tabular Q-learning algorithm to cooperatively balance bikes in a city.…”

Section: B Microagent Behavioral Interventionmentioning

confidence: 99%

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Zhang

Valencia

Chang

2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Agent-based modeling (ABM) involves developing models in which agents make adaptive decisions in a changing environment. Machine-learning (ML) based inference models can improve sequential decision-making by learning agents' behavioral patterns. With the aid of ML, this emerging area can extend traditional agent-based schemes that hardcode agents' behavioral rules into an adaptive model. Even though there are plenty of studies that apply ML in ABMs, the generalized applicable scenarios, frameworks, and procedures for implementations are not well addressed. In this article, we provide a comprehensive review of applying ML in ABM based on four major scenarios, i.e., microagent-level situational awareness learning, microagent-level behavior intervention, macro-ABM-level emulator, and sequential decision-making. For these four scenarios, the related algorithms, frameworks, procedures of implementations, and multidisciplinary applications are thoroughly investigated. We also discuss how ML can improve prediction in ABMs by trading off the variance and bias and how ML can improve the sequential decision-making of microagent and macrolevel policymakers via a mechanism of reinforced behavioral intervention. At the end of this article, future perspectives of applying ML in ABMs are discussed with respect to data acquisition and quality issues, the possible solution of solving the convergence problem of reinforcement learning, interpretable ML applications, and bounded rationality of ABM.

show abstract

Section: B Microagent Behavioral Interventionmentioning

confidence: 99%

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Zhang

Valencia

Chang

2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…Distributed Q-learning was used by Ebell et al [62] to facilitate energy sharing among households where the agents were only aware of their own actions but received a common reward. An edge computing architecture for energy sharing between smart houses was presented by Albataineh et al [63] that also used the decision tree learning method to calculate the electricity usage by each edge.…”

Section: Demand Predictionmentioning

confidence: 99%

Distributed Learning Applications in Power Systems: A Review of Methods, Gaps, and Challenges

Gholizadeh

Musilek

2021

Energies

View full text Add to dashboard Cite

In recent years, machine learning methods have found numerous applications in power systems for load forecasting, voltage control, power quality monitoring, anomaly detection, etc. Distributed learning is a subfield of machine learning and a descendant of the multi-agent systems field. Distributed learning is a collaboratively decentralized machine learning algorithm designed to handle large data sizes, solve complex learning problems, and increase privacy. Moreover, it can reduce the risk of a single point of failure compared to fully centralized approaches and lower the bandwidth and central storage requirements. This paper introduces three existing distributed learning frameworks and reviews the applications that have been proposed for them in power systems so far. It summarizes the methods, benefits, and challenges of distributed learning frameworks in power systems and identifies the gaps in the literature for future studies.

show abstract

“…To reduce computational complexity and improve robustness, multi-agent reinforcement learning (MARL) is introduced where energy devices [20] or production resourses [21] regarded as multiple agents can perceive the environment and independently adjust their energy policies to achieve the optimal performance [22]. Different constraints and diversity of energy are the main factors for multiple agents to make decisions in the energy management problem.…”

Section: Introductionmentioning

confidence: 99%

Energy Management Based on Multi-Agent Deep Reinforcement Learning for A Multi-Energy Industrial Park

Zhu,

Yang,

Liu

et al. 2022

Preprint

View full text Add to dashboard Cite

Owing to large industrial energy consumption, industrial production has brought a huge burden to the grid in terms of renewable energy access and power supply. Due to the coupling of multiple energy sources and the uncertainty of renewable energy and demand, centralized methods require large calculation and coordination overhead. Thus, this paper proposes a multi-energy management framework achieved by decentralized execution and centralized training for an industrial park. The energy management problem is formulated as a partiallyobservable Markov decision process, which is intractable by dynamic programming due to the lack of the prior knowledge of the underlying stochastic process.The objective is to minimize long-term energy costs while ensuring the demand of users. To solve this issue and improve the calculation speed, a novel multiagent deep reinforcement learning algorithm is proposed, which contains the following key points: counterfactual baseline for facilitating contributing agents to learn better policies, soft actor-critic for improving robustness and exploring

show abstract

Sharing of Energy Among Cooperative Households Using Distributed Multi-Agent Reinforcement Learning

Cited by 11 publications

References 10 publications

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Distributed Learning Applications in Power Systems: A Review of Methods, Gaps, and Challenges

Energy Management Based on Multi-Agent Deep Reinforcement Learning for A Multi-Energy Industrial Park

Contact Info

Product

Resources

About