Deep Reinforcement Learning with Immersion- and Invariance-based State Observer Control of Wave Energy Converters

Abstract

Composable life under the extensive global warming of the Earth encourages the progress of renewable energy devices and the adoption of new technologies, such as artificial intelligence. Regarding enormous potential of wave energy and its consistency, wave energy converter (WEC) plays vital role in uniform energy harvesting field. In this paper, the significant environmental changes in the ocean prompt us to propose an intelligent feedback control system to mitigate the impact of disturbances and variable wind effects on the efficacy of WECs. Deep reinforcement learning (DRL), as a powerful machine intelligence technique, is capable of identifying WECs as black-box models. Therefore, based on the DRL model, the disturbance and unmeasured state variables are simultaneously estimated in the extended state observer section. Leakage in identification data and real-time application requirements of limited number of layers in the deep neural networks are compensated by implementation of immersion and invariance-based extended state observer which improves coping with the unwanted exogenous noises as well. In the overall intelligent control system, the estimated parameters are inputted into the DRL as the actor-critic networks. The initial actor network is responsible for predicting the control action, while the subsequent critic network determines the decision criterion for evaluating the accuracy of the actor's estimated amount. Next, the output value of the critic stage is backpropagated through the layers to update the network weights. The simulation test results in MATLAB indicate the convergence of unmeasured parameters/states to the corresponding true values and the significance of newly designed intelligent DRL method.