Online Learning Algorithms

Smale, Steve; Yao, Yuan

doi:10.1007/s10208-004-0160-z

Cited by 118 publications

(87 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[15,18,23,25,29,37,39]). These developments, motivated by complexity theory in continuous optimization [26], concerned the convergence properties of SA-type methods during a finite number of iterations.…”

mentioning

confidence: 99%

Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms

2013

View full text Add to dashboard Cite

In this paper we present a generic algorithmic framework, namely, the accelerated stochastic approximation (AC-SA) algorithm, for solving strongly convex stochastic composite optimization (SCO) problems. While the classical stochastic approximation (SA) algorithms are asymptotically optimal for solving differentiable and strongly convex problems, the AC-SA algorithm, when employed with proper stepsize policies, can achieve optimal or nearly optimal rates of convergence for solving different classes of SCO problems during a given number of iterations. Moreover, we investigate these AC-SA algorithms in more detail, such as, establishing the large-deviation results associated with the convergence rates and introducing efficient validation procedure to check the accuracy of the generated solutions.

show abstract

mentioning

confidence: 99%

Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms

2013

View full text Add to dashboard Cite

show abstract

“…Expression (1) corresponds to the classical principle of minimum disturbance, and the constraint (2) sets to zero the a posteriori error. Though a large class of adaptive filtering techniques can be used here, we restrict ourselves to a gradient descent approach as studied in [10] and we consider the updating step…”

Section: Online Learning With Kernelsmentioning

confidence: 99%

“…This reduces the model order from m to m − 1. The Gram matrix K m−1 is obtained from K m by considering expression (10), where the latter matrix is arranged in order that its last column and row have entries relative to x n . Using the notation…”

Section: Incremental and Decremental Stepsmentioning

confidence: 99%

Functional estimation in Hilbert space for distributed learning in wireless sensor networks

Honeiné

Richard

Bermudez

et al. 2009

2009 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

In this paper, we propose a distributed learning strategy in wireless sensor networks. Taking advantage of recent developments on kernel-based machine learning, we consider a new sparsification criterion for online learning. As opposed to previously derived criteria, it is based on the estimated error and is therefore is well suited for tracking the evolution of systems over time. We also derive a gradient descent algorithm, and we demonstrate its relevance to estimate the dynamic evolution of temperature in a given region.

show abstract

“…Choice for the parameters of differential privacy and utilities analysis are presented for algorithms such as implicit gradient descent and generalized infinitesimal gradient descent. In this paper, the line of work begins with [8] is considered which can be thought as a kernel online learning algorithm.…”

Section: Introductionmentioning

confidence: 99%

A Study on Differential Private Online Learning

Nie¹,

Wang²

2017

JCC

View full text Add to dashboard Cite

Online learning algorithms are very attractive, in which iterations are applied efficiently instead of solving some optimization problems. In this paper, online learning with protecting privacy is considered. A perturbation term is added into the classical online algorithms to obtain the differential privacy property. Firstly the distribution for the perturbation term is deduced, and then an error analysis for the new algorithms is performed, which shows the convergence and learning rate. From the error analysis, a choice for the parameters for differential privacy can be found theoretically.

show abstract

Online Learning Algorithms

Cited by 118 publications

References 25 publications

Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms

Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms

Functional estimation in Hilbert space for distributed learning in wireless sensor networks

A Study on Differential Private Online Learning

Contact Info

Product

Resources

About