Reinforcement learning for optimal energy management of a solar microgrid

Leo, R; Milton, Robert; Sibi, S

doi:10.1109/ghtc-sas.2014.6967580

Cited by 30 publications

(18 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, few publications study on the energy management of the HRES. Kuznetsova (2013) proposed a two step ahead Q-learning method for defining the battery scheduling in a wind system, while Leo, Milton, and Sibi (2014) [32] developed a-three-step-ahead Q-learning for controlling the battery in a solar system. A novel online energy management technique using RL was developed in reference [33], which can learn and give the minimum power consumption without prior information on the workload.…”

Section: The Assessment Of the Energy Management System For Hresmentioning

confidence: 99%

Control Strategy of a Hybrid Renewable Energy System Based on Reinforcement Learning Approach for an Isolated Microgrid

Phan¹,

Lai²

2019

Applied Sciences

View full text Add to dashboard Cite

Due to the rising cost of fossil fuels and environmental pollution, renewable energy (RE) resources are currently being used as alternatives. To reduce the high dependence of RE resources on the change of weather conditions, a hybrid renewable energy system (HRES) is introduced in this research, especially for an isolated microgrid. In HRES, solar and wind energies are the primary energy resources while the battery and fuel cells (FCs) are considered as the storage systems that supply energy in case of insufficiency. Moreover, a diesel generator is adopted as a back-up system to fulfill the load demand in the event of a power shortage. This study focuses on the development of HRES with the combination of battery and hydrogen FCs. Three major parts were considered including optimal sizing, maximum power point tracking (MPPT) control, and the energy management system (EMS). Recent developments and achievements in the fields of machine learning (ML) and reinforcement learning (RL) have led to new challenges and opportunities for HRES development. Firstly, the optimal sizing of the hybrid renewable hydrogen energy system was defined based on the Hybrid Optimization Model for Multiple Energy Resources (HOMER) software for the case study in an island in the Philippines. According to the assessment of EMS and MPPT control of HRES, it can be concluded that RL is one of the most emerging optimal control solutions. Finally, a hybrid perturbation and observation (P&O) and Q-learning (h-POQL) MPPT was proposed for a photovoltaic (PV) system. It was conducted and validated through the simulation in MATLAB/Simulink. The results show that it showed better performance in comparison to the P&O method.

show abstract

Section: The Assessment Of the Energy Management System For Hresmentioning

confidence: 99%

Control Strategy of a Hybrid Renewable Energy System Based on Reinforcement Learning Approach for an Isolated Microgrid

Phan¹,

Lai²

2019

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…This binary representation allows one to take into account the charge/discharge constraint, Equation (3). The binary representation of the actions is shown in Figure 4 and Table 1.…”

Section: Microgrid Case Studymentioning

confidence: 99%

“…This has led to the smart grid paradigm, with technological advancement towards a green, intelligent and more efficient power grid. Microgrids can be a good base for the study and implementation of smart grid solutions [3][4][5]. Microgrids are electrical systems consisting of loads and distributed energy resources (like energy storage facilities and RES) that can operate in parallel with or disconnected from the main utility grid [6].…”

Section: Introductionmentioning

confidence: 99%

“…Ernst et al [12] suggest that RL and model predictive control can complement each other to achieve robust and accurate optimal control. Q-learning, a temporal difference method and a popular on-line RL algorithm, has been applied in energy management in microgrids [3,18,19]. A bias-corrected Q-learning algorithm for efficient operation of energy storage devices during variations in electricity spot prices is proposed in the work of Lee et al [20].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Battery Energy Management in a Microgrid Using Batch Reinforcement Learning

et al. 2017

View full text Add to dashboard Cite

Abstract:Motivated by recent developments in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL in energy management in microgrids. We tackle the challenge of finding a closed-loop control policy to optimally schedule the operation of a storage device, in order to maximize self-consumption of local photovoltaic production in a microgrid. In this work, the fitted Q-iteration algorithm, a standard batch RL technique, is used by an RL agent to construct a control policy. The proposed method is data-driven and uses a state-action value function to find an optimal scheduling plan for a battery. The battery's charge and discharge efficiencies, and the nonlinearity in the microgrid due to the inverter's efficiency are taken into account. The proposed approach has been tested by simulation in a residential setting using data from Belgian residential consumers. The developed framework is benchmarked with a model-based technique, and the simulation results show a performance gap of 19%. The simulation results provide insight for developing optimal policies in more realistically-scaled and interconnected microgrids and for including uncertainties in generation and consumption for which white-box models become inaccurate and/or infeasible.

show abstract

“…Q-learning is utilised for setting coordination control objectives and accelerating convergence characteristic [14]. On the other hand, Q-learning can optimise the power scheduling and increase the economy [15,16]. In [17], a fuzzy Q-learning algorithm based on MAS is utilised for coordinating the work of each component of MG which shares the state variables in MG.…”

Section: Introductionmentioning

confidence: 99%

Distributed cooperative control method based on network topology optimisation in microgrid cluster

Dou

Yue

et al. 2020

IET Renewable Power Generation

View full text Add to dashboard Cite

When several microgrids are interconnected into microgrid cluster (MGC), the network topology of MGC system would be more complex. Traditional hierarchical control based on droop control tends to make full use of all networks so that the costs of energy exchange could be high. Considering the voltage stability and power sharing of MGC system, a distributed cooperative control method based on network topology optimisation for MGC is proposed to control the power flow of multimicrogrid system. Specially, preliminary Q-leaning algorithm is utilised with establishment of new adjacency matrices for the distributed cooperative control to decrease the costs. Moreover, multi-level controls based on consensus protocol are proposed in the distributed cooperative MGC system to eliminate the necessity of a central controller. The adaptive regulation based on the state of charge among the battery groups will be taken as the first-level control. The second-level control including the current controller, voltage controller and power controller is implemented among the DGs and a novel third-level control based on consensus is applied in eliminating the voltage deviation caused by the first-level control and the second-level control. The effectiveness of the proposed method is verified through the simulation by simulink.

show abstract

Reinforcement learning for optimal energy management of a solar microgrid

Cited by 30 publications

References 12 publications

Control Strategy of a Hybrid Renewable Energy System Based on Reinforcement Learning Approach for an Isolated Microgrid

Control Strategy of a Hybrid Renewable Energy System Based on Reinforcement Learning Approach for an Isolated Microgrid

Battery Energy Management in a Microgrid Using Batch Reinforcement Learning

Distributed cooperative control method based on network topology optimisation in microgrid cluster

Contact Info

Product

Resources

About