Effective Backpropagation Training with Variable Stepsize

Magoulas, George D.; Vrahatis, Michael N.; Androulakis, G. S.

doi:10.1016/s0893-6080(96)00052-4

Cited by 166 publications

(92 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This issue concerns the training phase of an MLP and it has effectively been tackled in Magoulas et al (1997).…”

Section: Discussionmentioning

confidence: 99%

Solving the linear interval tolerance problem for weight initialization of neural networks

Adam¹,

Karras²,

Magoulas³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

Determining good initial conditions for an algorithm used to train a neural network is considered a parameter estimation problem dealing with uncertainty about the initial weights. Interval Analysis approaches model uncertainty in parameter estimation problems using intervals and formulating tolerance problems. Solving a tolerance problem is defining lower and upper bounds of the intervals so that the system functionality is guaranteed within predefined limits. The aim of this paper is to show how the problem of determining the initial weight intervals of a neural network can be defined in terms of solving a linear interval tolerance problem. The proposed Linear Interval Tolerance Approach copes with uncertainty about the initial weights without any previous knowledge or specific assumptions on the input data as required by approaches such as fuzzy sets or rough sets. The proposed method is tested on a number of well known benchmarks for neural networks trained with the back-propagation family of algorithms. Its efficiency is evaluated with regards to standard performance measures and the results obtained are compared against results of a number of well known and established initialization methods. These results provide credible evidence that the proposed method outperforms classical weight initialization methods.

show abstract

“…This issue concerns the training phase of an MLP and it has effectively been tackled in Magoulas et al (1997).…”

Section: Discussionmentioning

confidence: 99%

Solving the linear interval tolerance problem for weight initialization of neural networks

Adam¹,

Karras²,

Magoulas³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

show abstract

“…And also different methods have been done for progressing speed of training and its accuracy. You can find some of these achieved progresses have been applied in our suggestion in [10,11,12,13,14,15].…”

Section: Backpropagation Neural Networkmentioning

confidence: 99%

A Fast and Efficient Route Finding Method for Car Navigation Systems with Neural Networks

Hashemzadeh

2006

2006 10th IEEE International Enterprise Distributed Object Computing Conference (EDOC'06)

View full text Add to dashboard Cite

show abstract

“…Adaptive gradient-based algorithms with individual step-sizes try to overcome the inherent difficulty of choosing the right learning rates for each region of the search space depending on the application (Magoulas et al, 1997(Magoulas et al, , 1999. This is done by controlling the weight update of each weight in order to minimize oscillations and maximize the length of the step-size.…”

Section: Introductionmentioning

confidence: 99%

Sign-based learning schemes for pattern classification

Anastasiadis

Magoulas

Vrahatis

2005

Pattern Recognition Letters

View full text Add to dashboard Cite

This paper introduces a new class of sign-based training algorithms for neural networks that combine the sign-based updates of the Rprop algorithm with the composite nonlinear Jacobi method. The theoretical foundations of the class are described and a heuristic Rprop-based Jacobi algorithm is empirically investigated through simulation experiments in benchmark pattern classification problems. Numerical evidence shows that this new modification of the Rprop algorithm exhibits improved learning speed in all cases tested, and compares favorably against the Rprop and a recently proposed modification, the improved Rprop.

show abstract

Effective Backpropagation Training with Variable Stepsize

Cited by 166 publications

References 23 publications

Solving the linear interval tolerance problem for weight initialization of neural networks

Solving the linear interval tolerance problem for weight initialization of neural networks

A Fast and Efficient Route Finding Method for Car Navigation Systems with Neural Networks

Sign-based learning schemes for pattern classification

Contact Info

Product

Resources

About