A comparison of activation functions in multilayer neural network for predicting the production and consumption of electricity power

Abstract

Predicting electricity power is an important task, which helps power utilities in improving their systems’ performance in terms of effectiveness, productivity, management and control. Several researches had introduced this task using three main models: engineering, statistical and artificial intelligence. Based on the experiments, which used artificial intelligence models, multilayer neural networks model has proven its success in predicting many evaluation datasets. However, the performance of this model depends mainly on the type of activation function. Therefore, this paper introduces an experimental study for investigating the performance of the multilayer neural networks model with respect to different activation functions and different depths of hidden layers. The experiments in this paper cover the comparison among eleven activation functions using four benchmark electricity datasets. The activation functions under examination are sigmoid, hyperbolic tangent, SoftSign, SoftPlus, ReLU, Leak ReLU, Gaussian, ELU, SELU, Swish and Adjust-Swish. Experimental results show that ReLU and Leak ReLU activation functions outperform their counterparts in all datasets.

Neural network modeling of agglomeration firing process for polymetallic ores

“…The math sample is based on the solving of the conjugate question of heat transfer in the gas-charge-agglomerate system. The method of discrete performance of border conditions was taken as a mathematical method of modeling [28], [29]. In Figure 1 is presented the algorithm for solving the studied optimization problem which includes 9 stages.…”

“…It is given the opportunity to determine the specific outputs and the percentage of combustion products. The realization of this algorithm is made by a set of special computer programs [29], [30].…”

“…Another paper [6] is demonstated a comparative framework where a suite of long short-term memory (LSTM) recurrent neural networks (RNN) models are implemented to address the existing gaps and limitations of reported wind power forecasting methodologies. Similarly, Salam et al [29] proposed in his research the hybrid SVM-ANN net model. The performance of support vector machine (SVM), ANN, SVM with reduced features, and hybrid SVM-ANN model in the detection of breast cancer have been compared.…”

A deep learning approach based on stochastic gradient descent and least absolute shrinkage and selection operator for identifying diabetic retinopathy

“…The artificial intelligence techniques are widely used for classification and prediction in many applications [1]- [5]. In medical diagnosis, the deep learning models are becoming more popular for identifying various diseases [6], [7].…”