Tuning the Structure and Parameters of a Neural Network by Using Hybrid Taguchi-Genetic Algorithm

Abstract

In this paper, a hybrid Taguchi-genetic algorithm (HTGA) is applied to solve the problem of tuning both network structure and parameters of a feedforward neural network. The HTGA approach is a method of combining the traditional genetic algorithm (TGA), which has a powerful global exploration capability, with the Taguchi method, which can exploit the optimum offspring. The Taguchi method is inserted between crossover and mutation operations of a TGA. Then, the systematic reasoning ability of the Taguchi method is incorporated in the crossover operations to select the better genes to achieve crossover, and consequently enhance the genetic algorithms. Therefore, the HTGA approach can be more robust, statistically sound, and quickly convergent. First, the authors evaluate the performance of the presented HTGA approach by studying some global numerical optimization problems. Then, the presented HTGA approach is effectively applied to solve three examples on forecasting the sunspot numbers, tuning the associative memory, and solving the XOR problem. The numbers of hidden nodes and the links of the feedforward neural network are chosen by increasing them from small numbers until the learning performance is good enough. As a result, a partially connected feedforward neural network can be obtained after tuning. This implies that the cost of implementation of the neural network can be reduced. In these studied problems of tuning both network structure and parameters of a feedforward neural network, there are many parameters and numerous local optima so that these studied problems are challenging enough for evaluating the performances of any proposed GA-based approaches. The computational experiments show that the presented HTGA approach can obtain better results than the existing method reported recently in the literature.

Metaheuristic design of feedforward neural networks: A review of two decades of research

“…Such examples are as follows: in [209], a PSO-PSO method was proposed, in which a PSO (inner PSO block) was applied for optimizing weights that were nested under another PSO (outer PSO block) which was applied for optimizing the architecture of FNN by adding or deleting hidden node. Similarly, in [210,211], a hybrid Taguchi-genetic algorithm was proposed for optimizing the FNN architecture and weights, where authors used a genetic representation of the weights, but they select structure using constructive method (by adding hidden nodes one-by-one). A multidimensional PSO approach was proposed in [212] for constructing FNN automatically by using an architectural (topological) space.…”

Hybrid Robust Optimization for the Design of a Smartphone Metal Frame Antenna

“…However, it is very difficult to find a global optimum using only TM because an OA consists of a limited number and limited level sizes of design variables [4]. In order to resolve the drawbacks of TM, hybrid methods have been introduced [5][6][7]. In one study [5], a hybrid method which combined TM with an evolution strategy (ES) was proposed.…”

“…Subsequently, based on the result acquired from the ES, a robust antenna configuration is obtained by TM. TM with a genetic algorithm (GA) or particle swarm optimization (PSO) was also investigated in schemes where robust genes or particles were selected and utilized [6,7].…”

Designing multilayer perceptrons using a Guided Saw-tooth Evolutionary Programming Algorithm

“…In the past decade, Evolutionary Algorithms (EAs) (Back et al 1997;Eiben and Smith 2003) and Artificial Neural Networks (ANNs) (Bishop 2006) have been combined as a key research area, providing an interesting platform for simultaneously optimizing both the weights and the architecture of Multilayer Perceptrons (MLPs) (Tsai et al 2006;Leung et al 2003;Yao 1999;Maniezzo 1993) while avoiding the shortcomings of traditional Backpropagation (Angeline et al 1994). Evolutionary Programming (EP) was originally proposed by Fogel et al (1966) and it is one of the most renowned branches of EAs whose main variation operator is mutation.…”