Random Neural Network Learning Heuristics

Abstract

The random neural network (RNN) is a probabilitsic queueing theory-based model for artificial neural networks, and it requires the use of optimization algorithms for training. Commonly used gradient descent learning algorithms may reside in local minima, evolutionary algorithms can be also used to avoid local minima. Other techniques such as artificial bee colony (ABC), particle swarm optimization (PSO), and differential evolution algorithms also perform well in finding the global minimum but they converge slowly. The sequential quadratic programming (SQP) optimization algorithm can find the optimum neural network weights, but can also get stuck in local minima. We propose to overcome the shortcomings of these various approaches by using hybridized ABC/PSO and SQP. The resulting algorithm is shown to compare favorably with other known techniques for training the RNN. The results show that hybrid ABC learning with SQP outperforms other training algorithms in terms of mean-squared error and normalized root-mean-squared error.

RNN-ABC: A New Swarm Optimization Based Technique for Anomaly Detection

“…The following methods were adopted for training: RNN-IDS with a gradient descent algorithm (GD), and RNN-ABC with an artificial bee colony algorithm (ABC). In addition to this all, methods for both ABC and GD were trained using variant learning rates of 0.4, 0.1, and 0.01, respectively, and additional comparisons were undertaken to discuss the effects of mean square error (MSE) by analysis of mean of MSE (MMSE), standard deviation of MSE (SDMSE), best mean squared error (BMSE), and worst mean squared error (WMSE) [30] during the RNN training and testing phases.…”

Random Neural Network Methods and Deep Learning

“…It is reported in [142,143] that this algorithm achieved better performance than the gradient-descent algorithm in a combinatorial optimization problem emerging in disaster management. Javed et.al [96] proposed to combine the artificial bee colony/particle swarm optimization with the sequential quadratic programming optimization algorithm to train the RNN. The RNN was investigated for DL, and efficient DL algorithms based on the RNN were proposed [66,156].…”