Bayesian Optimized Echo State Network Applied to Short-Term Load Forecasting

Abstract

Load forecasting impacts directly financial returns and information in electrical systems planning. A promising approach to load forecasting is the Echo State Network (ESN), a recurrent neural network for the processing of temporal dependencies. The low computational cost and powerful performance of ESN make it widely used in a range of applications including forecasting tasks and nonlinear modeling. This paper presents a Bayesian optimization algorithm (BOA) of ESN hyperparameters in load forecasting with its main contributions including helping the selection of optimization algorithms for tuning ESN to solve real-world forecasting problems, as well as the evaluation of the performance of Bayesian optimization with different acquisition function settings. For this purpose, the ESN hyperparameters were set as variables to be optimized. Then, the adopted BOA employs a probabilist model using Gaussian process to find the best set of ESN hyperparameters using three different options of acquisition function and a surrogate utility function. Finally, the optimized hyperparameters are used by the ESN for predictions. Two datasets have been used to test the effectiveness of the proposed forecasting ESN model using BOA approaches, one from Poland and another from Brazil. The results of optimization statistics, convergence curves, execution time profile, and the hyperparameters’ best solution frequencies indicate that each problem requires a different setting for the BOA. Simulation results are promising in terms of short-term load forecasting quality and low error predictions may be achieved, given the correct options settings are used. Furthermore, since there is not an optimal global optimization solution known for real-world problems, correlations among certain values of hyperparameters are useful to guide the selection of such a solution.

Privacy‐preserving peak time forecasting with Learning to Rank XGBoost and extensive feature engineering

“…Especially time efficiency played a role in selecting this approach as the time for obtaining a prediction and, thus, also for hyperparameter tuning was limited in the competition the approach was developed for. Furthermore, several studies in the load forecasting field have shown that more accurate predictions can be reached by using Bayesian optimised model parameters [54][55][56].…”

PLS-CNN-BiLSTM: An End-to-End Algorithm-Based Savitzky–Golay Smoothing and Evolution Strategy for Load Forecasting

“…PLS is a straightforward dimensionality reduction technique that maps the variables in a new feature space with lower dimensions. The Variable Importance of load Patterns (VIP) for 32 features is shown in Regarding Figure 9, the most important features are hour, workday, temperature and lagged load (t − x) with x ∈ [1,2,3,4,5,6,7,11,12,13,17,18,19,20,21,22,23]. Thus, the selected threshold is VIP = 0.5.…”

“…Many techniques have been employed to improve the forecasting accuracy of the STLF [3][4][5]. Most efforts in the area of LF have been focused on the application of Machine learning (ML) models due to their mature development, large popularity, and ease of implementation [6].…”

A hybrid deep learning model for short‐term load forecasting of distribution networks integrating the channel attention mechanism

“…It's worth noting that the shortcoming of gradient vanishing during RNN training weakens the capability to capture long-term information. To address this issue, long short-term memory (LSTM) network, echo state network (ESN) and gated recurrent unit (GRU) network as the optimized versions of traditional RNN are employed to perform load forecasting in [19][20][21], respectively. In particular, GRU simplifies three unit functions of LSTM and ESN into two unit functions to fully decrease the number of model parameters, reducing the risk of model overfitting.…”