A comparison of gradient ascent, gradient descent and genetic‐algorithm‐based artificial neural networks for the binary classification problem

Abstract

We compare log maximum likelihood gradient ascent, root-mean-square error minimizing gradient descent and genetic-algorithm-based artificial neural network procedures for a binary classification problem. We use simulated data and real-world data sets, and four different performance metrics of correct classification, sensitivity, specificity and reliability for our comparisons. Our experiments indicate that a genetic-algorithmbased artificial neural network that maximizes the total number of correct classifications generally fares well for the binary classification problem. However, if the training data set contains inconsistent decisions or noise then the log maximum likelihood maximizing gradient ascent may be the best classification approach to use. The root-mean-square minimizing gradient descent approach appears to overfit training data and has the lowest reliability among the approaches considered for our research. At the end of the paper, we provide a few guidelines, including computational complexity, for selection of an appropriate technique for a given binary classification problem.

Feature selection for support vector machine in financial crisis prediction: a case study in China

“…Statistical methods include multiple discriminant analysis (MDA), logit regression and probit regression (Beaver, 1966;Altman, 1968;Ohlson, 1980). Artificial intelligence methods include casebased reasoning, inductive learning, artificial neural networks and knowledge-based expert systems (Dillard & Mutchker, 1987;Chung & Tam, 1992;Wong et al, 1995;Bryant, 1997;Pendharkar, 2007). Recently, the support vector machine (SVM) (Vapnik, 1998) has been introduced to financial applications, such as credit scoring, financial decisions and bankruptcy prediction (Tay & Cao, 2001;Shin et al, 2005;Gestel et al, 2006;Wu et al, 2007;Ding et al, 2008;Tsai, 2008).…”

Enhancing Scheduling Performance for a Wafer Fabrication Factory: The Biobjective Slack-Diversifying Nonlinear Fluctuation-Smoothing Rule

“…For details refer to Chen [29] and Pendharkar [30]. To accelerate convergence, a momentum term can be added to the learning expressions.…”

Statistical learning in chip (SLIC)

“…This is especially true in non-stationarity systems where the performance data becomes stale long before sufficient data has been collected to build a reasonable model. For these scenarios, a "model free" approach can be implemented based on gradient ascent [17]. The only information stored by the forward model will be an estimate of the gradient derived from the most recent data points.…”