Abstract. In this work a modification was made on the algorithm of Artificial Neural Networks (NN) Training of the Multilayer Perceptron type (MLP) based on multi-objective optimization (MOBJ), to increase its computational efficiency. Usually, the number of efficient solutions to be generated is a parameter that must be provided by the user. In this work, this number is automatically determined by an algorithm, through the usage of golden section, being generally less when specified, showing a sensible reduction in the processing time and keeping the high generalization capability of the obtained solution from the original method.

No abstract

PurposeThis work presents a new learning scheme for improving generalization of Multilayer Perceptrons (MLPs). The proposed Multi-objective algorithm (MOBJ) approach minimizes both the sum of squared error and the norm of network weight vectors to obtain the Pareto-optimal solutions [1]. Preliminar results are shown in [3].Since the Pareto-optimal solutions are not unique, we need a decision phase in order to choose the best one as a final solution by using a validation set. The final solution is expected to balance network variance and bias [2] and, as a result, generates a solution with high generalization capacity, avoiding over and underfitting. Novel aspects of the workThe proposed multi-objective method controls model flexibility independently of the number of network weights, although a minimal number of weights is needed. Also, the training parameters produce minor effects on the final solution and, consequently, tuning the best set of training parameters is an easy task. MethodsThe proposed algorithm are based on Multi-objective techniques [1]. The constraint problem ȯ[1] is used to approach the learning task. Also, the efficiency of the decision phase "Decisor" is proved. ConclusionsThe classification and regression MOBJ solutions are compared with Weight decay, Optimal Brain Damage, Early Stopping, 10-Fold Cross-Validation, Support Vector Machines and Backpropagation. It is concluded that the proposed method is able to generate high generalization solutions and its operation is simple and efficient.