Analysis and Application of Grey Wolf Optimizer-Long Short-Term Memory

Abstract

Long short-term memory (LSTM) is widely applied in both academic and industrial fields. However, there is no reliable criterion on selecting hyperparameters of LSTM. Currently, although some widely used classic methods such as random search and grid search have obtained success to some extent, the problems in local optimum and convergence still exist. In this research, we propose to use grey wolf optimizer (GWO) to search for the hyperparameters of LSTM. Through the method, the superiority of metaheuristic in global optimization and the strength of LSTM in predicting are combined. In this model, number of hidden layer nodes and learning rate of LSTM are set as preys, and grey wolf pack has a simple but efficient mechanism to search for the optimal hyperparameters. The benchmark tests on several basic functions were utilized, and the results were verified by a comparative study with random search, support vector regression and several other regression methods. Specifically, we applied this algorithm in predicting the degradation trend of the airborne fuel pump. As a result, the ergodicity and convergence of the algorithm are proved mathematically based on Markov processes theory. The benchmark tests show that the GWO-LSTM model holds for predicting data with low overall slope and high partial fluctuation. The application in airborne fuel pump shows that, trained by dataset with 5700 points, the proposed model could predict sequence of 300 points with root mean square error 0.617 after 30 iterations of optimizing, which is 2.512 previously. The result further demonstrates that the proposed algorithm is applicable to make prediction with high accuracy. Overall, the effectiveness of GWO-LSTM model is verified from theoretical proof to benchmark tests and then to actual product application.

Hybrid the long short-term memory with whale optimization algorithm and variational mode decomposition for monthly evapotranspiration estimation

“…Generally, constructing the hybrid ML models is a complex and time-consuming procedure 6,14-18 . However, the hybrid ML models have higher accuracy than the physically-based methods [20][21][22][23] . Therefore, using the hybrid ML models to precisely estimate the ET is a core issue in hydrology and ecology.…”

“…overcome the drawbacks of ML models, meta-heuristic algorithms such as flower pollination algorithm (FPA) 6 , firefly algorithm (FFA) 11 , intelligent water drops (IWD) algorithm 12 , whale optimization algorithm (WOA) 18 , grey wolf optimizer algorithm (GWO) 19,20 etc., were employed to determine the optimal hyperparameters of ML models. Studies have shown that ML models coupled with meta-heuristic algorithms have higher computing performance than that of single ML models and physically-based methods 12,16,18,21,22 .…”

Optimal coordinated generation scheduling considering day‐ahead PV and wind power forecast uncertainty

“…In that regard, a GWO is employed to optimize the LSTM hyperparameter and to build an encouraging input data regression model. The application of GWO-based LSTM to predict the degradation trend of the airborne fuel pump is employed in [32]. It is worth mentioning that the performance of deep learning models is data-dependent and tuning the hyperparameter values improves the performance of data prediction.…”

“…In this study, a robust GWO is considered to ensure that the optimal scheduled solution can survive under a certain degree of variability under uncertain conditions in the scheduling problem decision variables and renewable forecasts. The optimization problem given for the deterministic UC-ELD formulation is reformulated by (32) considering an additional constraint that represents the normalized variation of the objective function and then handled by using a barrier function if the robustness threshold is not maintained. min…”

A Diagnosis Framework for High‐reliability Equipment with Small Sample Based on Transfer Learning

“…In the main loop, the fuel pump pumps the oil from the oil feeding tank to the oil storage tank. For cycling, the oil in the storage tank returns to the feeding tank by gravity through the valve [39]. rough cycling, the working environment of the test pump is stable.…”