Maximum Power Point Tracker Controller for Solar Photovoltaic Based on Reinforcement Learning Agent with a Digital Twin

Abstract

Photovoltaic (PV) energy, representing a renewable source of energy, plays a key role in the reduction of greenhouse gas emissions and the achievement of a sustainable mix of energy generation. To achieve the maximum solar energy harvest, PV power systems require the implementation of Maximum Power Point Tracking (MPPT). Traditional MPPT controllers, such as P&O, are easy to implement, but they are by nature slow and oscillate around the MPP losing efficiency. This work presents a Reinforcement learning (RL)-based control to increase the speed and the efficiency of the controller. Deep Deterministic Policy Gradient (DDPG), the selected RL algorithm, works with continuous actions and space state to achieve a stable output at MPP. A Digital Twin (DT) enables simulation training, which accelerates the process and allows it to operate independent of weather conditions. In addition, we use the maximum power achieved in the DT to adjust the reward function, making the training more efficient. The RL control is compared with a traditional P&O controller to validate the speed and efficiency increase both in simulations and real implementations. The results show an improvement of 10.45% in total power output and a settling time 24.54 times faster in simulations. Moreover, in real-time tests, an improvement of 51.45% in total power output and a 0.25 s settling time of the DDPG compared with 4.26 s of the P&O is obtained.

A Review on Digital Twins and Its Application in the Modeling of Photovoltaic Installations

“…Applying this technology to a solar PV system will allow for the monitoring, control and management of the renewable energy generated and consumed by the PV installation [31][32][33][34], as well as fault detection by comparing the output of the DT and that of the PV system [22,[34][35][36]. This would enable the optimisation of photovoltaic renewable energy production [37,38].…”

“…This section compares the use of a Digital Twin (DT) for the Maximum Power Point Tracking (MPPT) in [37,38].…”

“…In ref. [38], it is mentioned that P&O algorithms require very low computational power but have some disadvantages, such as oscillation around the MPP point, which leads to a loss of efficiency, partial shadow conditions (PSC), the use of P&O algorithms [47,48], etc. In ref.…”

“…In ref. [38], it is shown that a large computational cost is required when using the incremental conductance (InCond) algorithms that look for voltage and current variations instead of power variations. However, it should be mentioned that these algorithms are fast and accurate.…”

“…The Reinforcement Learning (RL) technique is one of the most widely used techniques for MPPT controllers [38] because it is resistant to environmental variations [54]. Currently there are different types of RL agents.…”