Stagewise Newton, differential dynamic programming, and neighboring optimum control for neural-network learning

Mizutani, Eiji; Dreyfus, Stuart E.

doi:10.1109/acc.2005.1470149

Cited by 11 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stage-wise Newton (SN) method [26] and differential dynamic programming (DDP) [27], [28] are the two standard methods for this purpose. The pure SN method [26] sequentially optimizes a second-order model of the Hamiltonian H k px k , u k , λ k`1 q " L k px k , u k q`λ T k`1 f k px k , u k q while DDP sequentially optimizes a local model of the Hamilton-Jacobi-Bellman value function V k px k , u k q defined recursively by Vk px k q " min…”

Section: Stage-wise Newton and Differential Dynamic Programmingmentioning

confidence: 99%

Discrete optimal control on lie groups and applications to robotic vehicles

Kobilarov

2014

2014 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

This paper is concerned with optimal trajectory generation for robotic multi-body systems. The focus is on discrete optimal control methods which operate intrinsically in the state space system manifold and do not require coordinate charts or projections. This is accomplished by defining both the dynamics and the optimal control solution as sequences of vector fields mapping to curves on the Lie group through retraction maps, and defining variations and differentiation with respect to such vector fields. As a result, standard trajectory optimization methods can be easily extended to the Lie group setting without loss of efficiency. The methods are illustrated with three numerical examples: a quadrotor, an aerial vehicle with manipulators, and a simple nonholonomic system.

show abstract

Section: Stage-wise Newton and Differential Dynamic Programmingmentioning

confidence: 99%

Discrete optimal control on lie groups and applications to robotic vehicles

Kobilarov

2014

2014 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…On the other hand, such nice sparsity may disappear when weight-sharing and weight-pruning are applied (as usual in optimal control [8]) so that all the terminal parameters 0N-IN are shared among the terminal states yN. In this way, MLPlearning exhibits a great deal of structure.…”

Section: Discussionmentioning

confidence: 96%

“…denotes a differentiable state-transition, function of nonlinear dynamics, and x1, the "before-node" net input to node j at layer s, depends on only a subset of all decisions taken at stage s-1. In spite of this distinction and others, using a vector of states as a basic ingredient allows us to adopt analogous formulas available in the optimal control theory (see [8]). The key concept behind the theory resides in stagewise implementation; in fact, first-order BP is essentially a simplified stagewise optimalcontrol gradient formula developed in early 1960s [6].…”

mentioning

confidence: 99%

“…INTRODUCTION In multi-stage optimal control problems, second-order optimization procedures (see [8] and references therein) proceed in a stagewise manner since N, the number of stages, is often very large. Naturally, those methods can be employed for optimizing multi-stage feed-forward neural networks: In this paper, we focus on an N-layered multilayer perceptron (MLP), which gives rise to an N-stage decision making problem.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Second-order backpropagation algorithms for a stagewise-partitioned separable Hessian matrix

Mizutani

Dreyfus²,

Demmel³

Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005.

Self Cite

View full text Add to dashboard Cite

Recent advances in computer technology allow the implementation of some important methods that were assigned lower priority in the past due to their computational burdens. Second-order backpropagation (BP) is such a method that computes the exact Hessian matrix of a given objective function. We describe two algorithms for feed-forward neural-network (NN) learning with emphasis on how to organize Hessian elements into a so-called stagewise-partitioned block-arrow matrix form:(1) stagewise BP, an extension of the discrete-time optimal-control stagewise Newton of Dreyfus 1966; and (2) nodewise BP, based on direct implementation of the chain rule for differentiation attributable to Bishop 1992. The former, a more systematic and cost-efficient implementation in both memory and operation, progresses in the same layer-by-layer (i.e., stagewise) fashion as the widely-employed first-order BP computes the gradient vector. We also show intriguing separable structures of each block in the partitioned Hessian, disclosing the rank of blocks.I. INTRODUCTION In multi-stage optimal control problems, second-order optimization procedures (see [8] and references therein) proceed in a stagewise manner since N, the number of stages, is often very large. Naturally, those methods can be employed for optimizing multi-stage feed-forward neural networks: In this paper, we focus on an N-layered multilayer perceptron (MLP), which gives rise to an N-stage decision making problem. At each stage s, we assume there are P8 (s = 1, *,N) states (or nodes) and n8 (s = 1, --, N-1) decision parameters (or weights), denoted by an n,-vector 08,8+1 (between layers s and s+1). No decisions are to be made at terminal stage N (or layer N); hence, the N-1 decision stages in total. To compute the gradient vector for optimization purposes, we employ the "first-order" backpropagation (BP) process [5], [6], [7], which consists of two major procedures: forward pass and backward pass [see later Eq. (2)]. A forward-pass situation in MLPlearning, where the node outputs in layer s -1 (denoted by yS-1) affect the node outputs in the next layer s (i.e., y8) via connection parameters (denoted by 081s, between those two layers), can be interpreted as a situation in optimal control where state y8-1 at stage s-1 is moved to state y8 at the next stage s by decisions 081,8. In the backward pass, sensitivities of the objective function E with respect to states (i.e., node sensitivities) are propagated from one stage back to another while computing gradients and Hessian elements. However, MLPs exhibit a great deal of structure, which turns out to be a very special case in optimal control; for instance, the "afternode" outputs (or states) are evaluated individually at each stage as yj = fj'(x^), where f(.) denotes a differentiable state-transition, function of nonlinear dynamics, and x1, the "before-node" net input to node j at layer s, depends on only a subset of all decisions taken at stage s-1. In spite of this distinction and others, using a vector of states as a basic ingr...

show abstract

“…Learning with multilayer-perceptron (MLP) neural networks is a multiple N -stage decision-making (or discretetime optimal control) problem [1]. The optimal-control theory available in control engineering dictates to us a variety of elaborate learning schemes in the context of N -layered MLP-learning with P s nodes at layer s (s = 1, ..., N ).…”

Section: Introductionmentioning

confidence: 99%

On derivation of stagewise second-order backpropagation by invariant imbedding for multi-stage neural-network learning

Mizutani¹,

Dreyfus²

The 2006 IEEE International Joint Conference on Neural Network Proceedings

Self Cite

View full text Add to dashboard Cite

We present a simple, intuitive argument based on "invariant imbedding" in the spirit of dynamic programming to derive a stagewise second-order backpropagation (BP) algorithm. The method evaluates the Hessian matrix of a general objective function efficiently by exploiting the multistage structure embedded in a given neural-network model such as a multilayer perceptron (MLP). In consequence, for instance, our stagewise BP can compute the full Hessian matrix "faster" than the standard method that evaluates the GaussNewton Hessian matrix alone by rank updates in nonlinear least squares learning. Through our derivation, we also show how the procedure serves to develop advanced learning algorithms; in particular, we explain how the introduction of "stage costs" leads to alternative systematic implementations of multi-task learning and weight decay.

show abstract

Stagewise Newton, differential dynamic programming, and neighboring optimum control for neural-network learning

Cited by 11 publications

References 21 publications

Discrete optimal control on lie groups and applications to robotic vehicles

Discrete optimal control on lie groups and applications to robotic vehicles

Second-order backpropagation algorithms for a stagewise-partitioned separable Hessian matrix

On derivation of stagewise second-order backpropagation by invariant imbedding for multi-stage neural-network learning

Contact Info

Product

Resources

About