A new interactive model for improving the learning performance of back propagation neural network

Wang, Ching-Hwang; Kao, Chih-Han; Lee, Wei-Hsien

doi:10.1016/j.autcon.2006.12.007

Cited by 15 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Many different types of methods were developed to overcome the local‐minimum problem of the backpropagation algorithm. One obvious approach is to concentrate on optimization of learning rates or step sizes30–33 and the employment of various minimization methodologies such as conjugate gradient,34, 35 Levenberg‐Marguardt algorithm,36, 37 stochastic backpropagation,38 genetic algorithm,39, 40 simulated annealing,41, 42 or a hybrid of optimization methods 43. The second approach focuses on optimizing network architecture during training by employing genetic algorithm,44–47 self‐organized network,48 or fuzzy logic 49–51.…”

Section: Introductionmentioning

confidence: 99%

Improving the prediction accuracy of residue solvent accessibility and real‐value backbone torsion angles of proteins by guided‐learning through a two‐layer neural network

2008

View full text Add to dashboard Cite

This paper attempts to increase the prediction accuracy of residue solvent accessibility and real-value backbone torsion angles of proteins through improved learning. Most methods developed for improving the backpropagation algorithm of artificial neural networks are limited to small neural networks. Here, we introduce a guided-learning method suitable for networks of any size. The method employs a part of the weights for guiding and the other part for training and optimization. We demonstrate this technique by predicting residue solvent accessibility and real-value backbone torsion angles of proteins. In this application, the guiding factor is designed to satisfy the intuitive condition that for most residues, the contribution of a residue to the structural properties of another residue is smaller for greater separation in the protein-sequence distance between the two residues. We show that the guided-learning method makes a 2-4% reduction in ten-fold cross-validated mean absolute errors (MAE) for predicting residue solvent accessibility and backbone torsion angles, regardless of the size of database, the number of hidden layers and the size of input windows. This together with introduction of two-layer neural network with a bipolar activation function leads to a new method that has a MAE of 0.11 for residue solvent accessibility, 36° for ψ, and 22° for ϕ. The method is available as a Real-SPINE 3.0 server in http://sparks.informatics.iupui.edu.

show abstract

Section: Introductionmentioning

confidence: 99%

Improving the prediction accuracy of residue solvent accessibility and real‐value backbone torsion angles of proteins by guided‐learning through a two‐layer neural network

2008

View full text Add to dashboard Cite

show abstract

“…To address these problems effectively, many improved approaches have been proposed. For example, approaches such as too early saturation avoidance, weight adjusting, normalisation, interaction, learning rate changing and stimulation function changing (Yu and Chen 1997;Yam and Chow 2000;Trentin 2001;Magoulas, Plagiananakos, and Vrahatis 2002;Eom, Jung, and Sirisena 2003;Zweiri, Whidborne, and Seneviratne 2003;Yan and Jue 2004;Zhang 2006Zhang , 2008Wang, Kao, and Lee 2007). All these approaches have made some improvement to the performance of BP.…”

Section: Introductionmentioning

confidence: 93%

A kind of BP neural network algorithm based on grey interval

Luo

Zhou

et al. 2011

International Journal of Systems Science

View full text Add to dashboard Cite

In order to improve the learning ability of a forward neural network, in this article, we incorporate the feedback back-propagation (FBBP) and grey system theory to consider the learning and training of a neural network new perspective. By reducing the input grey degree we optimise the input of the neural network to make it more rational for learning and training of neural networks. Simulation results verified the efficiency of the proposed algorithm by comparing its performance with that of FBBP and classic back-propagation (BP). The results showed that the proposed algorithm has the characteristics of fast training and strong ability of generalisation and it is an effective learning method.

show abstract

“…There are basically on selection of dynamic variation of learning rate and momentum, selection of better activation function and better cost function. In 2007, Wang et al [20] has proposed the individual inference adjusting learning rate technique (IIALR) to enhance the learning performance of the BP network. The mechanism of the weight adjustment in the IIALR is an individual learning rate for each weight.…”

Section: Research Trends Of Bp Learningmentioning

confidence: 99%

Study of cost functions in Three Term Backpropagation for classification problems

Shamsuddin

Alwee

Kuppusamy

et al. 2009

2009 World Congress on Nature &Amp; Biologically Inspired Computing (NaBIC)

View full text Add to dashboard Cite

Three Term Backpropagation(BP) Network as proposed by Zweiri in 2003 has outperformed standard Two Term Backpropagation. However, further studies on Three Term Backpropagation in 2007 indicated that this network only surpassed standard BP for small scale datasets but not for medium and large scale datasets. It has also been observed that by using Mean Square Error (MSE) as a cost function in Three Term BP has some drawbacks, and these include incorrect saturation and tend to trap in local minima, resulting in slow convergence and poor performance. In this study, thorough experiments on implementing various cost functions are executed to probe the effectiveness of Three Term BP network. The cost functions under investigations include Mean Square Error (MSE), Bernoulli function, Modified cost function andImproved cost function. The results reveal that MSE is not an ideal cost function to be used for Three Term BP. Hence, alternative cost functions need to be considered when using BP network for classification problems.

show abstract

A new interactive model for improving the learning performance of back propagation neural network

Cited by 15 publications

References 20 publications

Improving the prediction accuracy of residue solvent accessibility and real‐value backbone torsion angles of proteins by guided‐learning through a two‐layer neural network

Improving the prediction accuracy of residue solvent accessibility and real‐value backbone torsion angles of proteins by guided‐learning through a two‐layer neural network

A kind of BP neural network algorithm based on grey interval

Study of cost functions in Three Term Backpropagation for classification problems

Contact Info

Product

Resources

About