Evolutionary Based Weight Decaying Method for Neural Network Training

Tsoulos, Ioannis G.; Tzallas, Alexandros T.; Tsalikakis, Dimitrios G.

doi:10.1007/s11063-017-9660-0

Cited by 3 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The paper by Geman et al [51] as well the article by Hawkins [52] thoroughly discussed the topic of overfitting. Examples of techniques proposed to tackle this problem are the weight sharing methods [53,54], methods that reduce the number of parameters of the model (pruning methods) [55,56], weight elimination [57][58][59], weight decaying methods [60,61], dropout methods [62,63], the Sarprop method [64], and positive correlation methods [65]. Recently, a variety of papers have proposed methods to handle the overfitting problem in various cases, such as the usage of genetic algorithms for training data selection in RBF networks [66], the evolution of RBF models using genetic algorithms for rainfall prediction [67], and pruning decision trees using genetic algorithms [68,69].…”

mentioning

confidence: 99%

A Feature Construction Method That Combines Particle Swarm Optimization and Grammatical Evolution

Tsoulos

Tzallas

2023

Applied Sciences

View full text Add to dashboard Cite

The problem of data classification or data fitting is widely applicable in a multitude of scientific areas, and for this reason, a number of machine learning models have been developed. However, in many cases, these models present problems of overfitting and cannot generalize satisfactorily to unknown data. Furthermore, in many cases, many of the features of the input data do not contribute to learning, or there may even be hidden correlations between the features of the dataset. The purpose of the proposed method is to significantly reduce data classification or regression errors through the usage of a technique that utilizes the particle swarm optimization method and grammatical evolution. This method is divided into two phases. In the first phase, artificial features are constructed using grammatical evolution, and the progress of the creation of these features is controlled by the particle swarm optimization method. In addition, this new technique utilizes penalty factors to limit the generated features to a range of values to make training machine learning models more efficient. In the second phase of the proposed technique, these features are exploited to transform the original dataset, and then any machine learning method can be applied to this dataset. The performance of the proposed method was measured on some benchmark datasets from the relevant literature. Also, the method was tested against a series of widely used machine learning models. The experiments performed showed a significant improvement of 30% on average in the classification datasets and an even greater improvement of 60% in the data fitting datasets.

show abstract