Backpropagation through time: what it does and how to do it

Werbos, Paul J.

doi:10.1109/5.58337

Cited by 4,012 publications

(1,764 citation statements)

References 11 publications

Supporting

Mentioning

1,722

Contrasting

Unclassified

Order By: Relevance

“…(7) by using BPTT (Werbos, 1990). BPTT efficiently calculates the gradient of J( x 0 , w) with respect to the weight vector of the action network, w, for a given trajectory with arbitrary initial state x 0 .…”

Section: Backpropagation Through Time Algorithmmentioning

confidence: 99%

“…These variables hold the "ordered partial derivatives" of J with respect to the given variable name, so that for example J x k = ∂ + J ∂ x k . This ordered partial derivative, as defined by Werbos (Werbos, 1990;Werbos et al, 1992), represents the derivative of J with respect to x k , assuming all other variables which depend upon x k in lines 3-7 of Alg. 1 are not fixed, and thus their derivatives will influence the value of ∂ + J ∂ x k via the chain rule.…”

Section: Backpropagation Through Time Algorithmmentioning

confidence: 99%

“…1 are not fixed, and thus their derivatives will influence the value of ∂ + J ∂ x k via the chain rule. The derivation of the gradient computation part of the algorithm (lines 8-16) is exact, using the method known as generalized backpropagation (Werbos, 1990), or automatic-differentiation (Werbos, 2005;Rall, 1981). For full details of this derivation, see Werbos et al (1992).…”

Section: Backpropagation Through Time Algorithmmentioning

confidence: 99%

“…To improve on this situation further, we are using an architecture that uses an action network only, but which is trained off-line through backpropagation through time (BPTT) (Werbos, 1990). This approach has the advantage that the learning algorithm is true gradient descent on the cost-to-go function, and so convergence is assured (assuming a smooth error minimization surface, and a sufficiently small learning rate).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances

Fairbank¹,

Li²,

Fu³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

This is the accepted version of the paper.This version of the publication may differ from the final published version. Permanent repository link AbstractWe present a recurrent neural-network (RNN) controller designed to solve the tracking problem for control systems. We demonstrate that a major difficulty in training any RNN is the problem of exploding gradients, and we propose a solution to this in the case of tracking problems, by introducing a stabilization matrix and by using carefully constrained context units. This solution allows us to achieve consistently lower training errors, and hence allows us to more easily introduce adaptive capabilities. The resulting RNN is one that has been trained off-line to be rapidly adaptive to changing plant conditions and changing tracking targets.The case study we use is a renewable-energy generator application; that of producing an efficient controller for a three-phase grid-connected converter. The controller we produce can cope with random variation of system parameters and fluctuating grid voltages. It produces tracking control with almost instantaneous response to changing reference states, and virtually zero oscil-$ This work was supported in part by the U.S. National Science Foundation under Grant EECS 1059265/1102159, the Mary K. Finley Missouri Endowment, and the Missouri S&T Intelligent Systems Center.Email addresses: michael.fairbank@virgin.net (Michael Fairbank), sli@eng.ua.edu (Shuhui Li), xfu@crimson.ua.edu (Xingang Fu), E.Alonso@city.ac.uk (Eduardo Alonso), dwunsch@mst.edu (Donald Wunsch) Preprint submitted to Neural Networks September 23, 2013 lation. This compares very favorably to the classical proportional integrator (PI) controllers, which we show produce a much slower response and settling time. In addition, the RNN we propose exhibits better learning stability and convergence properties, and can exhibit faster adaptation, than is achievable with adaptive critic designs.

show abstract

Section: Backpropagation Through Time Algorithmmentioning

confidence: 99%

Section: Backpropagation Through Time Algorithmmentioning

confidence: 99%

Section: Backpropagation Through Time Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances

Fairbank¹,

Li²,

Fu³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

show abstract

“…To update the model parameters, we have used Truncated Back-Propagation-Through-Time (T-BPTT) (Werbos, 1990) (Werbos, 1990) with stochastic gradient descent. We fixed the depth of BPTT to 7 for all the models.…”

Section: Implementation Detailsmentioning

confidence: 99%

Translation Quality Estimation using Recurrent Neural Network

Patel¹,

Sasikumar²

2016

Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers

View full text Add to dashboard Cite

This paper describes our submission to the shared task on word/phrase level Quality Estimation (QE) in the First Conference on Statistical Machine Translation (WMT16).The objective of the shared task was to predict if the given word/phrase is a correct/incorrect (OK/BAD) translation in the given sentence. In this paper, we propose a novel approach for word level Quality Estimation using Recurrent Neural Network Language Model (RNN-LM) architecture. RNN-LMs have been found very effective in different Natural Language Processing (NLP) applications. RNN-LM is mainly used for vector space language modeling for different NLP problems. For this task, we modify the architecture of RNN-LM. The modified system predicts a label (OK/BAD) in the slot rather than predicting the word. The input to the system is a word sequence, similar to the standard RNN-LM. The approach is language independent and requires only the translated text for QE. To estimate the phrase level quality, we use the output of the word level QE system.

show abstract