Logistic Regression by Means of Evolutionary Radial Basis Function Neural Networks

Abstract

Abstract-This paper proposes a hybrid multilogistic methodology, named logistic regression using initial and radial basis function (RBF) covariates. The process for obtaining the coefficients is carried out in three steps. First, an evolutionary programming (EP) algorithm is applied, in order to produce an RBF neural network (RBFNN) with a reduced number of RBF transformations and the simplest structure possible. Then, the initial attribute space (or, as commonly known as in logistic regression literature, the covariate space) is transformed by adding the nonlinear transformations of the input variables given by the RBFs of the best individual in the final generation. Finally, a maximum likelihood optimization method determines the coefficients associated with a multilogistic regression model built in this augmented covariate space. In this final step, two different multilogistic regression algorithms are applied: one considers all initial and RBF covariates (multilogistic initial-RBF regression) and the other one incrementally constructs the model and applies cross validation, resulting in an automatic covariate selection [simplelogistic initial-RBF regression (SLIRBF)]. Both methods include a regularization parameter, which has been also optimized. The methodology proposed is tested using 18 benchmark classification problems from well-known machine learning problems and two real agronomical problems. The results are compared with the corresponding multilogistic regression methods applied to the initial covariate space, to the RBFNNs obtained by the EP algorithm, and to other probabilistic classifiers, including different RBFNN design methods [e.g., relaxed variable kernel density estimation, support vector machines, a sparse classifier (sparse multinomial logistic regression)] and a procedure similar to SLIRBF but using product unit basis functions. The SLIRBF models are found to be competitive when compared with the corresponding multilogistic regression methods and the RBFEP method. A measure of statistical significance is used, which indicates that SLIRBF reaches the state of the art.

A Fast Adaptive Tunable RBF Network For Nonstationary Systems

“…Unlike our previous work in [15], the structure of the new node in this paper is not optimized by the PSO algorithm, or any other evolutionary algorithms [16].…”

Radial basis function neural networks: a topical state-of-the-art survey

“…Viewing the RBF network training procedure as an optimization problem, one realizes that the objective function usually presents some rather unwelcome properties including, multimodality, non-di erentiability and high levels of noise. As these characteristics make use of standard optimization methods ine cient, it is no surprise that a signi cant number of studies have focused on optimizing the RBF training procedure through the use of alternative approaches, such as evolutionary-based computation techniques [68]. The resulting methodologies include a genetic algorithm for optimizing the number and coordinates of RBF centers [69], a hybrid multi-logistic methodology applying evolutionary programming for producing RBFs with simpler structures [29], a multi-objective evolutionary algorithm to optimize RBF networks including some new genetic operators in the evolutionary process [71], and an evolutionary algorithm that performs feature and model selection simultaneously for RBF classi ers in reduced computational times [72].…”

When Neural Network Computation Meets Evolutionary Computation: A Survey

“…To improve the performance of the NN by optimizing its structure, some of the papers encode all of the information about NN topology in each individual in EC algorithms [11,19,[21][22][23][24][25][26], as the way we do in Sect. 2.1, while others just encode some of the parameters which determines the structure of the neural network such as number of hidden nodes [29], corresponding centers [30], radii [14,30].…”

“…It is one of the core and challenging tasks in machine learning and statistics, EC-NNs and EC-NNs-FUZZ systems are diffusely employed to pattern classification [30], classification of semiconductor defects [10], brain-computer interface classification [17], hypoglycemia detection system [19] and power transformer differential protection [31]. Moreover, some NNs are used to optimized the classification problem, for instance, [11] proposes a new multi classification algorithm using multilayer perceptron neural network models, and [23] proposes a novel approach to improve the classification performance.…”