Metaheuristics for the feedforward artificial neural network (ANN) architecture optimization problem

Carvalho, Adenilson R.; Ramos, Fernando M.; Chaves, Antônio Augusto

doi:10.1007/s00521-010-0504-3

Cited by 61 publications

(42 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, local search algorithms like SA, TS, GRAP, VNS [98], estimations of distribution algorithm [187] and global search algorithms like GA, ACO, and memetic algorithm, were examined thoroughly in [186]. Additionally, many researchers studied the performance of metaheuristic algorithms for the training of the FNN and reported that the metaheuristic approaches outperform all the conventional methods by a huge margin [188][189][190].…”

Section: Weight Optimizationmentioning

confidence: 99%

Metaheuristic design of feedforward neural networks: A review of two decades of research

Ojha

Abraham

Snel

2017

Engineering Applications of Artificial Intelligence

457

154

View full text Add to dashboard Cite

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era.

show abstract

Section: Weight Optimizationmentioning

confidence: 99%

Metaheuristic design of feedforward neural networks: A review of two decades of research

Ojha

Abraham

Snel

2017

Engineering Applications of Artificial Intelligence

457

154

View full text Add to dashboard Cite

show abstract

“…More recently, [129] has shown the benefits of optimizing each neuron's transfer function, creating heterogeneous networks. In a similar approach, [130] present a methodology to find the best architecture of a neural network using metaheuristics. The authors tested the following ones: generalized extremal optimization, VNS, SA, and canonical GA.…”

Section: Using Metaheuristics To Improve Machine Learningmentioning

confidence: 99%

Learnheuristics: hybridizing metaheuristics with machine learning for optimization with dynamic inputs

et al. 2017

View full text Add to dashboard Cite

This paper reviews the existing literature on the combination of metaheuristics with machine learning methods and then introduces the concept of learnheuristics, a novel type of hybrid algorithms. Learnheuristics can be used to solve combinatorial optimization problems with dynamic inputs (COPDIs). In these COPDIs, the problem inputs (elements either located in the objective function or in the constraints set) are not fixed in advance as usual. On the contrary, they might vary in a predictable (non-random) way as the solution is partially built according to some heuristic-based iterative process. For instance, a consumer's willingness to spend on a specific product might change as the availability of this product decreases and its price rises. Thus, these inputs might take different values depending on the current solution configuration. These variations in the inputs might require from a coordination between the learning mechanism and the metaheuristic algorithm: at each iteration, the learning method updates the inputs model used by the metaheuristic.

show abstract

“…The optimal neural network topology can be expressed to minimize a cost function. Metaheuristics used to compute the best topology include hybrid Particle Swarm Optimization (PSO) and Genetic Algorithms (GA) [22], Constructive Method and Pruning [5], or applying severe algorithms: a comparison among Genetic Algorithm (GA), Simulated Annealing (SA), Generalized Extreme Optimization (GEO), and Variable Neighbourhood Search (VNS) [3].…”

Section: Self-configured Mlp-nnmentioning

confidence: 99%

Kinetic Studies of Fenton Oxidation Reaction by UV-VIS Spectroscopy

Dai¹,

Flatberg²,

Preisig³

et al. 2018

jlce

View full text Add to dashboard Cite

Numerical weather and climate use sophisticated mathematical models. These models are employed to simulate the atmospheric dynamics to perform a medium-range forecasting and climate prediction. Such an approach allows to estimate all meteorological variables for a future time period: wind fields, air temperature, pressure, moisture, and precipitation field. Precipitation is one of the most difficult fields for prediction. The latter statement is verified due to high variability in space and time. However, precipitation is a key issue in many activities of society. An alternative approach for climate prediction to the precipitation field is to employ the Artificial Neural Network (ANN). Such technique has a reduced computational cost in comparison with time integration of the partial differential equations. One challenge to employ an ANN is to determine the topology or configuration of a neural network. Here, a supervised ANN is designed to perform the precipitation prediction looking at two different periods: monthly and seasonal precipitation. The method is applied to the Southern region of Brazil. The definition of the neural network topology is addressed as an optimization problem. The best configuration is computed by minimizing a cost function. The optimization problem is solved by a new meta-heuristic: Multi-Particle Collision Algorithm (MPCA). In addition, a technique based on rough set theory is used to reduce the data space dimension. The predicted precipitation is evaluated by comparison with measured data. The prediction is also evaluated using full and reduced input data for a neural predictive model.

show abstract

Metaheuristics for the feedforward artificial neural network (ANN) architecture optimization problem

Cited by 61 publications

References 19 publications

Metaheuristic design of feedforward neural networks: A review of two decades of research

Metaheuristic design of feedforward neural networks: A review of two decades of research

Learnheuristics: hybridizing metaheuristics with machine learning for optimization with dynamic inputs

Kinetic Studies of Fenton Oxidation Reaction by UV-VIS Spectroscopy

Contact Info

Product

Resources

About