Accelerating the convergence of the back-propagation method

Vogl, Thomas P.; Mangis, J. K.; Rigler, A. K.; Zink, Walter; Alkon, Daniel L.

doi:10.1007/bf00332914

Cited by 979 publications

(347 citation statements)

References 8 publications

Supporting

Mentioning

306

Contrasting

Unclassified

Order By: Relevance

“…Consequently, several researchers have devised modifications to the backpropagation algorithm to increase the convergence rate. The general approach has been to vary the learning rate dynamically during training in order to maintain it at the largest value that will not cause oscillations (13] (2]. Attempts have been made to learn from a subset of the patterns to determine the network size and initialize the weights to reduce training time [12].…”

Section: Introductionmentioning

confidence: 99%

An improved algorithm for neural network classification of imbalanced training sets

Anand

Mehrotra

Mohan

et al. 1993

IEEE Trans. Neural Netw.

227

108

View full text Add to dashboard Cite

In this paper, we analyze the reason for the slow rate of convergence of net output error when using the backpropagation algorithm to train neural networks for a two-class problems in which the numbers of exemplars for the two classes differ greatly. This occurs because the negative gradient vector computed by backpropagation for an imbalanced training set does not point initially in a downhill direction for the class with the smaller number of exemplars. Consequently, in the initial iteration, the net error for the exemplars in this class increases significantly. The subsequent rate of convergence of the net error is very low. We suggest a modified technique for calculating a direction in weight-space which is downhill for both classes. Using this algorithm, we have been able to accelerate the rate of learning for two-class classification problems by an order of magnitude.

show abstract

Section: Introductionmentioning

confidence: 99%

An improved algorithm for neural network classification of imbalanced training sets

Anand

Mehrotra

Mohan

et al. 1993

IEEE Trans. Neural Netw.

227

108

View full text Add to dashboard Cite

show abstract

“…In the present study, the time-series forecasts denoted by UWD k (t) were generated using activation functions g(X) described by the logarithmic sigmoid, ψ(X) and the output function, χ(X) equations [VOGL et al 1988] as seen in equations (7) and (8).…”

Section: Theoretical Overview Extreme Learning Machinementioning

confidence: 99%

Water demand forecasting using extreme learning machines

Tiwari¹,

Adamowski²,

Adamowski³

2016

Journal of Water and Land Development

View full text Add to dashboard Cite

The capacity of recently-developed extreme learning machine (ELM) modelling approaches in forecasting daily urban water demand from limited data, alone or in concert with wavelet analysis (W) or bootstrap (B) methods (i.e., ELM, ELM W , ELM B ), was assessed, and compared to that of equivalent traditional artificial neural network-based models (i.e., ANN, ANN W , ANN B ). The urban water demand forecasting models were developed using 3-year water demand and climate datasets for the city of Calgary, Alberta, Canada. While the hybrid ELM B and ANN B models provided satisfactory 1-day lead-time forecasts of similar accuracy, the ANN W and ELM W models provided greater accuracy, with the ELM W model outperforming the ANN W model. Significant improvement in peak urban water demand prediction was only achieved with the ELM W model. The superiority of the ELM W model over both the ANN W or ANN B models demonstrated the significant role of wavelet transformation in improving the overall performance of the urban water demand model.

show abstract

“…It is a simple, heuristic, strategy for accelerating the BP algorithm that is based on the use of a momentum term (BPM). The second method was suggested by Vogl [31]. It increases the convergence of the BP by adapting the learning rate at each epoch, in such a way that monotone decrease of the error is enforced (VMRZA).…”

Section: Training Mlps Using Back-propagation Algorithmsmentioning

confidence: 99%

Distributed computing methodology for training neural networks in an image-guided diagnostic application

Plagianakos

Magoulas

Vrahatis

2006

Computer Methods and Programs in Biomedicine

View full text Add to dashboard Cite

Distributed Computing is a process through which a set of computers connected by a network is used collectively to solve a single problem. In this paper, we propose a distributed computing methodology for training neural networks for the detection of lesions in colonoscopy. Our approach is based on partitioning the training set across multiple processors using a Parallel Virtual Machine. In this way, interconnected computers of varied architectures can be used for the distributed evaluation of the error function and gradient values, and, thus, training neural networks utilizing various learning methods. The proposed methodology has large granularity and low synchronization, and has been implemented and tested. Our results indicate that the Parallel Virtual Machine implementation of the training algorithms developed leads to considerable speedup, especially when large network architectures and training sets are used.

show abstract

Accelerating the convergence of the back-propagation method

Cited by 979 publications

References 8 publications

An improved algorithm for neural network classification of imbalanced training sets

An improved algorithm for neural network classification of imbalanced training sets

Water demand forecasting using extreme learning machines

Distributed computing methodology for training neural networks in an image-guided diagnostic application

Contact Info

Product

Resources

About