Convergence of exponentiated gradient algorithms

Hill, Sam; Williamson, Robert C.

doi:10.1109/78.923303

Cited by 28 publications

(12 citation statements)

References 15 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second point is that the term keeps the stepsizes of individual taps from going to zero as individual taps go to zero. The third point is that according to equations given in [6], [8], and [9], NSLMS and LMS have the same asymptotic MSE, so long as the stepsize µ is the same for both algorithms.…”

Section: System Model and Algorithmmentioning

confidence: 99%

“…(Note that there is some similarity between NSLMS and the EG ± algorithm in equations (3) and (4) of [6].) There is of course a disadvantage: if the optimal weight vector changes drastically, then the estimate will temporarily have the wrong impression of which taps are important.…”

Section: System Model and Algorithmmentioning

confidence: 99%

“…The alternate philosophy is to improve performance at the expense of retaining a possibly high complexity. For example, the Exponentiated Gradient (EG) algorithm [5] has recently been shown to have better performance than typical gradient methods when the target weight vector is sparse [6], [7], though it does not decrease the required complexity. Our approach in this paper is similar.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

NSLMS: a proportional weight algorithm for sparse adaptive filters

Martin

Johnson²

2001

Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256)

View full text Add to dashboard Cite

In this paper we discuss a proportional weight algorithm that is similar to LMS. The distinction is that the new algorithm (called normalized sparse LMS, or NSLMS) has a time-varying vector stepsize, whose coefficients are proportional to the magnitudes of the current values of the tap estimates.We show that when the system to be identified is sparse, NSLMS converges faster than LMS (to the same asymptotic MMSE for both algorithms). We also discuss the effect of the initialization on the performance of NSLMS.

show abstract

Section: System Model and Algorithmmentioning

confidence: 99%

Section: System Model and Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

NSLMS: a proportional weight algorithm for sparse adaptive filters

Martin

Johnson²

2001

Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256)

View full text Add to dashboard Cite

show abstract

“…Exponentiated gradient descent share similar convergence guarantees [9], but implements a different prior over the space of score functions. It places large prior weight on score functions with a great deal of dynamic range.…”

Section: Exponentiated Gradient Variantmentioning

confidence: 99%

Imitation Learning for Locomotion and Manipulation

Ratliff¹,

Bagnell²,

Srinivasa³

2007

View full text Add to dashboard Cite

Decision making in robotics often involves computing an optimal action for a given state, where the space of actions under consideration can potentially be large and state dependent. Many of these decision making problems can be naturally formalized in the multiclass classification framework, where actions are regarded as labels for states. One powerful approach to multiclass classification relies on learning a function that scores each action; action selection is done by returning the action with maximum score. In this work, we focus on two imitation learning problems in particular that arise in robotics. The first problem is footstep prediction for quadruped locomotion, in which the system predicts next footstep locations greedily given the current four-foot configuration of the robot over a terrain height map. The second problem is grasp prediction, in which the system must predict good grasps of complex free-form objects given an approach direction for a robotic hand. We present experimental results of applying a recently developed functional gradient technique for optimizing a structured margin formulation of the corresponding large non-linear multiclass classification problems.

show abstract

“…The concept of exponentiated gradient adaptation can be seen by considering, for example, the exponentiated gradient algorithm with unnormalized weights (EGUAE) which is given in [15,19,16] as…”

Section: Exponentiated Gradientmentioning

confidence: 99%

Adaptive algorithms for sparse echo cancellation

2006

View full text Add to dashboard Cite

The cancellation of echoes is a vital component of telephony networks. In some cases the echo response that must be identified by the echo canceller is sparse, as for example when telephony traffic is routed over networks with unknown delay such as packet-switched networks. The sparse nature of such a response causes standard adaptive algorithms including normalized LMS to perform poorly. This paper begins by providing a review of techniques that aim to give improved echo cancellation performance when the echo response is sparse. In addition, adaptive filters can also be designed to exploit sparseness in the input signal by using partial update procedures. This concept is discussed and the MMax procedure is reviewed. We proceed to present a new high performance sparse adaptive algorithm and provide comparative echo cancellation results to show the relative performance of the existing and new algorithms. Finally, an efficient low cost implementation of our new algorithm using partial update adaptation is presented and evaluated. This algorithm exploits both sparseness of the echo response and also sparseness of the input signal in order to achieve high performance without high computational cost. r

show abstract

Convergence of exponentiated gradient algorithms

Cited by 28 publications

References 15 publications

NSLMS: a proportional weight algorithm for sparse adaptive filters

NSLMS: a proportional weight algorithm for sparse adaptive filters

Imitation Learning for Locomotion and Manipulation

Adaptive algorithms for sparse echo cancellation

Contact Info

Product

Resources

About