Echo state network‐based Q‐learning method for optimal battery control of offices combined with renewable energy

Abstract

An echo state network (ESN)-based Q-learning method is developed for optimal energy management of an office, where the solar energy is introduced as the renewable source, and a battery is installed with a control unit. The energy consumption in the office, also considered as the energy demand, is separated into those from sockets, lights and airconditioners. First, ESNs, well known for their excellent modelling performance for time series, are employed to model the time series of the real-time electricity rate, renewable energy and energy demand as periodic functions. Second, given the periodic models of the electricity rate, renewable energy and energy demand, an ESN-based Q-learning method with the Q-function approximated by an ESN is developed and implemented to determine the optimal charging/discharging/idle strategies for the battery in the office, so that the total cost of electricity from the grid can be reduced. Finally, numerical analysis is conducted to illustrate the performance of the developed method.

Double‐deck optimal schedule of micro‐grid based on demand‐side response

“…However, owing to the demand and energy are presented as time-varying functions, it cannot availably approach the optimal Q-function and optimal control. To solve the problem, a dual iterative ADHDP algorithm is developed to optimise the battery operation in a small power system in [24,25]. Inspired by Shi et al [25], this algorithm is used for solving the optimal scheduling model of micro-grid that includes BESS.…”

“…Battery energy storage units are essential for micro-grid operation [25,28]. To increase battery efficiency and extend the battery's working life, some constraints need to be considered…”

“…Inspired by the authors in [24,25], the iterative process is derived to find the optimal function, consisting of the external iteration and the internal iteration. The external iteration aims to obtain the most optimal performance index function by comparing with other suboptimal function.…”

Optimal Energy Routing Design in Energy Internet with Multiple Energy Routing Centers Using Artificial Neural Network-Based Reinforcement Learning Method

“…In Reference [22], by designing a marketing auction mechanism, a reinforcement learning algorithm was adopted to obtain an energy management strategy with minimized economic cost. A study of energy management in an office building with renewable energy resources was carried out in Reference [23], and an echo-state based reinforcement learning method was used to manage the output power of devices in an office building. In these studies, only electrical power management was involved and only economical cost was considered.…”

Research on Marine Photovoltaic Power Forecasting Based on Wavelet Transform and Echo State Network

“…Build a more accurate neural network. Echo State Network [13,14,15] (ESN) first proposed by Professor Jaeger of Germany in 2001 .The main feature of the ESN is that the dynamic reserve pool composed of a large number of neurons randomly generated and sparsely associated with each other is regarded as the hidden layer of the network. Compared with the traditional neural network, ESN has more neurons, and neurons connected randomly, once generated will remain unchanged [16].The network effectively avoids the problem that the network structure is difficult to be determined and the burden of computing is too heavy [17].…”