A direct adaptive method for faster backpropagation learning: the RPROP algorithm

Riedmiller, Martin; Braun, Heinrich

doi:10.1109/icnn.1993.298623

Cited by 2,999 publications

(1,886 citation statements)

References 3 publications

Supporting

Mentioning

1,792

Contrasting

Unclassified

Order By: Relevance

“…This approach reduces the variance of the estimation. All runs were done using the RPROP training algorithm Riedmiller & Braun, 1993 using the squared error function and the parameters + = 1 : 1, , = 0 : 5, 0 2 0:05 : : : 0 : 2 randomly per weight, max = 50, min = 0, initial weights ,0:5 : : : 0 : 5 randomly. RPROP is a fast backpropagation variant similar in spirit to quickprop Fahlman, 1988 .…”

Section: Methodsmentioning

confidence: 99%

Automatic early stopping using cross validation: quantifying the criteria

Prechelt¹

1998

Neural Networks

778

375

View full text Add to dashboard Cite

Cross validation can be used to detect when over tting starts during supervised training of a neural network; training is then stopped before convergence to avoid the overtting early stopping" . The exact criterion used for cross validation based early stopping, however, is chosen in an ad-hoc fashion by most researchers or training is stopped interactively. To aid a more well-founded selection of the stopping criterion, 14 di erent automatic stopping criteria from 3 classes were evaluated empirically for their e ciency and e ectiveness in 12 di erent classi cation and approximation tasks using multi layer perceptrons with RPROP training. The experiments show that on the average slower stopping criteria allow for small improvements in generalization on the order of 4 , but cost about factor 4 longer training time.

show abstract

Section: Methodsmentioning

confidence: 99%

Automatic early stopping using cross validation: quantifying the criteria

Prechelt¹

1998

Neural Networks

778

375

View full text Add to dashboard Cite

show abstract

“…1. This gradient was accelerated by RPROP (Riedmiller and Braun, 1993) before each weight update was finally made. After 800 iterations, the average trajectory cost per time step along the whole trajectory was calculated and noted in Table 1, in the first row (without sta-bilization matrix).…”

Section: Training the Neural Controllermentioning

confidence: 99%

An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances

Fairbank¹,

Li²,

Fu³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

This is the accepted version of the paper.This version of the publication may differ from the final published version. Permanent repository link AbstractWe present a recurrent neural-network (RNN) controller designed to solve the tracking problem for control systems. We demonstrate that a major difficulty in training any RNN is the problem of exploding gradients, and we propose a solution to this in the case of tracking problems, by introducing a stabilization matrix and by using carefully constrained context units. This solution allows us to achieve consistently lower training errors, and hence allows us to more easily introduce adaptive capabilities. The resulting RNN is one that has been trained off-line to be rapidly adaptive to changing plant conditions and changing tracking targets.The case study we use is a renewable-energy generator application; that of producing an efficient controller for a three-phase grid-connected converter. The controller we produce can cope with random variation of system parameters and fluctuating grid voltages. It produces tracking control with almost instantaneous response to changing reference states, and virtually zero oscil-$ This work was supported in part by the U.S. National Science Foundation under Grant EECS 1059265/1102159, the Mary K. Finley Missouri Endowment, and the Missouri S&T Intelligent Systems Center.Email addresses: michael.fairbank@virgin.net (Michael Fairbank), sli@eng.ua.edu (Shuhui Li), xfu@crimson.ua.edu (Xingang Fu), E.Alonso@city.ac.uk (Eduardo Alonso), dwunsch@mst.edu (Donald Wunsch) Preprint submitted to Neural Networks September 23, 2013 lation. This compares very favorably to the classical proportional integrator (PI) controllers, which we show produce a much slower response and settling time. In addition, the RNN we propose exhibits better learning stability and convergence properties, and can exhibit faster adaptation, than is achievable with adaptive critic designs.

show abstract

“…Neural networks had five hidden neurons and were trained for 100 epochs with a resilient backpropagation algorithm. 36 The requirement for the partitioning algorithm was to provide sufficiently accurate partitioning in a reasonable time. Because OLS proved to be the fastest prediction method with overall prediction accuracy close to the other two methods, it was chosen for specialized predictors in the partitioning algorithm.…”

Section: Choice Of Specialized Predictorsmentioning

confidence: 99%

Flavors of protein disorder

et al. 2003

View full text Add to dashboard Cite

Intrinsically disordered proteins are characterized by long regions lacking 3-D structure in their native states, yet they have been so far associated with 28 distinguishable functions. Previous studies showed that protein predictors trained on disorder from one type of protein often achieve poor accuracy on disorder of proteins of a different type, thus indicating significant differences in sequence properties among disordered proteins.

show abstract

A direct adaptive method for faster backpropagation learning: the RPROP algorithm

Cited by 2,999 publications

References 3 publications

Automatic early stopping using cross validation: quantifying the criteria

Automatic early stopping using cross validation: quantifying the criteria

An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances

Flavors of protein disorder

Contact Info

Product

Resources

About