Lifelong control of off-grid microgrid with model-based reinforcement learning

Totaro, Simone; Jönsson, Anders; Cornélusse, Bertrand

doi:10.1016/j.energy.2021.121035

Cited by 27 publications

(7 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning-based control systems are also very popular for off-grid scenarios, as off-grid energy management systems are gaining increasing attention to provide sustainable and reliable energy services. In References [45] and [46], the authors developed algorithms based on deep reinforcement to deal with the uncertain and stochastic nature of renewable energy sources.…”

Section: Machinementioning

confidence: 99%

Renewable energy management in smart home environment via forecast embedded scheduling based on Recurrent Trend Predictive Neural Network

Nakıp

Çopur²,

Biyik

et al. 2023

Applied Energy

View full text Add to dashboard Cite

Section: Machinementioning

confidence: 99%

Renewable energy management in smart home environment via forecast embedded scheduling based on Recurrent Trend Predictive Neural Network

Nakıp

Çopur²,

Biyik

et al. 2023

Applied Energy

View full text Add to dashboard Cite

“…RL learns the optimal control policies from interactions with the environment and selects the actions based on a given reward mechanism [27]. The control policies can be adaptively adjusted based on the feedback from the environment and show an advantage in adapting to the stochastic environment changes [31]. Table 1 summarizes different applications of RL algorithms in managing energy system operations.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…However, real-world data with stochastic patterns generally lead to an unstable training process. Model-based algorithms take actions obtained based on a simple environment model, leading to low requirements for data and better generalization of convergence [31]. Most researchers have been attempting to optimize objectives (e.g., user cost reduction, renewable energy consumption, user comfort, and load flexibility) by implementing a single approach, such as the rule-based method, Q-learning algorithms, DQN, and other regulation strategies.…”

Section: Reinforcement Learningmentioning

confidence: 99%

Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control

et al. 2023

Energies

View full text Add to dashboard Cite

The development of distributed renewable energy resources and smart energy management are efficient approaches to decarbonizing building energy systems. Reinforcement learning (RL) is a data-driven control algorithm that trains a large amount of data to learn control policy. However, this learning process generally presents low learning efficiency using real-world stochastic data. To address this challenge, this study proposes a model-based RL approach to optimize the operation of existing zero-energy houses considering PV generation consumption and energy costs. The model-based approach takes advantage of the inner understanding of the system dynamics; this knowledge improves the learning efficiency. A reward function is designed considering the physical constraints of battery storage, photovoltaic (PV) production feed-in profit, and energy cost. Measured data of a zero-energy house are used to train and test the proposed RL agent control, including Q-learning, deep Q network (DQN), and deep deterministic policy gradient (DDPG) agents. The results show that the proposed RL agents can achieve fast convergence during the training process. In comparison with the rule-based strategy, test cases verify the cost-effectiveness performances of proposed RL approaches in scheduling operations of the hybrid energy system under different scenarios. The comparative analysis of test periods shows that the DQN agent presents better energy cost-saving performances than Q-learning while the Q-learning agent presents more flexible action control of the battery with the fluctuation of real-time electricity prices. The DDPG algorithm can achieve the highest PV self-consumption ratio, 49.4%, and the self-sufficiency ratio reaches 36.7%. The DDPG algorithm outperforms rule-based operation by 7.2% for energy cost during test periods.

show abstract

“…Kolodziejczyk et al [141] model the maximum charging and discharging power as non-linear functions of SoC. Totaro et al [95] model how charging and discharging efficiencies as well as the battery storage capacity degrade over time. For problem formulations that permit selling battery energy to the grid, the inverter efficiency as a function of discharging power is a significant factor taken into account only in the minority of the works [23].…”

Section: A Capturing Battery Losses In the Reinforcement Learning Env...mentioning

confidence: 99%

“…In the case of isolated microgrids, purchases from an external electricity market are either not possible [168], [95] or a last resort to complement local fossil-fuel based emergency generation [96]. Phan & Lai [169] and Zhang et al [96] note that the trend towards a decentralized electric power system should in some seashore regions be complemented with a move to decentralized freshwater production, so a desalination plant is added to the microgrid.…”

Section: B) Isolatedmentioning

confidence: 99%

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

2022

View full text Add to dashboard Cite

The transition to renewable production and smart grids is driving a massive investment to battery storages, and reinforcement learning (RL) has recently emerged as a potentially disruptive technology for their control and optimization of battery storage systems. A surge of papers has appeared in the last two years applying reinforcement learning to the optimization of battery storages in buildings, energy communities, energy harvesting Internet of Things networks, renewable generation, microgrids, electric vehicles and plug-in hybrid electric vehicles. This article reviews these applications through 4 different perspectives. Firstly, the type of optimization problem is analyzed; the literature can be divided to approaches that optimize either financial targets or energy efficiency. Secondly, the approaches for handling user comfort are analyzed for applications that may impact a human user. Thirdly, this paper discusses the approach to model and reduce battery degradation. Fourthly, the articles are categorized by application context and applications likely to attract a high amount of research are identified. The paper concludes with a list of unresolved challenges.

show abstract

Lifelong control of off-grid microgrid with model-based reinforcement learning

Cited by 27 publications

References 27 publications

Renewable energy management in smart home environment via forecast embedded scheduling based on Recurrent Trend Predictive Neural Network

Renewable energy management in smart home environment via forecast embedded scheduling based on Recurrent Trend Predictive Neural Network

Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

Contact Info

Product

Resources

About