Enhanced Two-Phase method in fast learning algorithms

Cheung, Chun Shun; Ng, Sin-Chun; Lui, Andrew K.; Xu, Sean Shensheng

doi:10.1109/ijcnn.2010.5596519

Cited by 14 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It takes 500,000 epochs. In [23] the limited error by less than 3 · 10 −4 . The convergence rate was very slow.…”

Section: Implementation Dbbplr Algorithmmentioning

confidence: 91%

Dynamically-adaptive Weight in Batch Back Propagation Algorithm via Dynamic Training Rate for Speedup and Accuracy Training

Al_Duais¹,

Mohamad²

2017

JTIT

View full text Add to dashboard Cite

Abstract-The main problem of batch back propagation (BBP) algorithm is slow training and there are several parameters need to be adjusted manually, such as learning rate. In addition, the BBP algorithm suffers from saturation training. The objective of this study is to improve the speed up training of the BBP algorithm and to remove the saturation training. The training rate is the most significant parameter for increasing the efficiency of the BBP. In this study, a new dynamic training rate is created to speed the training of the BBP algorithm. The dynamic batch back propagation (DBBPLR) algorithm is presented, which trains with adynamic training rate. This technique was implemented with a sigmoid function. Several data sets were used as benchmarks for testing the effects of the created dynamic training rate that we created. All the experiments were performed on Matlab. From the experimental results, the DBBPLR algorithm provides superior performance in terms of training, faster training with higher accuracy compared to the BBP algorithm and existing works.Keywords-artificial neural network (ANN), batch back propagation algorithm, dynamic training rate, speed up training, accuracy training.

show abstract

“…It takes 500,000 epochs. In [23] the limited error by less than 3 · 10 −4 . The convergence rate was very slow.…”

Section: Implementation Dbbplr Algorithmmentioning

confidence: 91%

Dynamically-adaptive Weight in Batch Back Propagation Algorithm via Dynamic Training Rate for Speedup and Accuracy Training

Al_Duais¹,

Mohamad²

2017

JTIT

View full text Add to dashboard Cite

show abstract

“…The main drawback of the SBP algorithm is slowing down training as it often takes along time to learn and get the desired results. However, the network can get stuck at a local minimum when, O r the output training of the output layer, approach the extremes of 1 or 0 [5].…”

Section: Introductionmentioning

confidence: 99%

Dynamic training rate for backpropagation learning algorithm

Al_Duais

Yaakub

Yusoff

2013

2013 IEEE 11th Malaysia International Conference on Communications (MICC)

View full text Add to dashboard Cite

In this paper, we created a dynamic function training rate for the Back propagation learning algorithm to avoid the local minimum and to speed up training. The Back propagation with dynamic training rate (BPDR) algorithm uses the sigmoid function. The 2-dimensional XOR problem and iris data were used as benchmarks to test the effects of the dynamic training rate formulated in this paper. The results of these experiments demonstrate that the BPDR algorithm is advantageous with regards to both generalization performance and training speed. The stop training or limited error was determined by1.0

show abstract

“…The Enhanced Two-Phase Method (E2P) is proposed in [22] to solve the local minimum problem and the error overshooting problem. E2P can effectively identify them when one of these two problems exists, and hence assign suitable fast learning algorithms to speed up the learning process with a better global convergence capability.…”

mentioning

confidence: 99%

Improving the Quickprop algorithm

Cheung¹,

Ng²,

Lui³

2012

The 2012 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

Backpropagation (BP) algorithm is the most popular supervised learning algorithm that is extensively applied in training feed-forward neural networks. Many BP modifications have been proposed to increase the convergence rate of the standard BP algorithm, and Quickprop is one the most popular fast learning algorithms. The convergence rate of Quickprop is very fast; however, it is easily trapped into a local minimum and thus it cannot converge to the global minimum. This paper proposes a new fast learning algorithm modified from Quickprop. By addressing the drawbacks of the Quickprop algorithm, the new algorithm has a systematic approach to improve the convergence rate and the global convergence capability of Quickprop. Our performance investigation shows that the proposed algorithm always converges with a faster learning rate compared with Quickprop. The improvement in the global convergence capability is especially large. In one learning problem (application), the global convergence capability increased from 4% to 100%. I. INTRODUCTION Backpropagation (BP) algorithm [1] is the most popular supervised learning algorithm that is extensively applied in training feed-forward neural networks. BP is successful because it is simple and its computational complexity is low. The learning rate of BP is slow, however, and it is easy to be trapped in a local minimum, especially for some non-linearly separable problems such as XOR problem [2, 3]. Some modifications have been proposed to address these drawbacks and enhance the performance of the standard BP algorithm, and most of those modifications have focused on handling the "flat spot" problem to speed up the learning process [12-22]. The main reason to have the "flat spot" problem is due to the premature saturation in the derivative of the activation function [4-7]. If a learning process is trapped into a "flat spot" or a local minimum, the weight update of the learning process will become very slow or even unchanged. Different fast learning algorithms use different way to solve this problem. Quickprop [12] predicts the location of a minimum and adjusts the weights accordingly to converge to the minimum by considering successive values of the gradient of the error surface. RPROP [13] is a variable step size algorithm that updates the weight to adapt the learning

show abstract

Enhanced Two-Phase method in fast learning algorithms

Cited by 14 publications

References 17 publications

Dynamically-adaptive Weight in Batch Back Propagation Algorithm via Dynamic Training Rate for Speedup and Accuracy Training

Dynamically-adaptive Weight in Batch Back Propagation Algorithm via Dynamic Training Rate for Speedup and Accuracy Training

Dynamic training rate for backpropagation learning algorithm

Improving the Quickprop algorithm

Contact Info

Product

Resources

About