First-order methods of smooth convex optimization with inexact oracle

Devolder, Olivier; Glineur, François; Nesterov, Yurii

doi:10.1007/s10107-013-0677-5

Cited by 413 publications

(550 citation statements)

References 12 publications

Supporting

Mentioning

527

Contrasting

Unclassified

Order By: Relevance

“…Paper [38] considers inexact accelerated hybrid extragradient-proximal methods, but actually the framework is shown to include only the case of the exact accelerated forward-backward algorithm. In [22], convergence rates for an accelerated projected-subgradient method is proved. The case of an exact projection step is considered, and the authors assume the availability of an oracle that yields global lower and upper bounds on the function.…”

Section: Main Contributionsmentioning

confidence: 99%

Accelerated and Inexact Forward-Backward Algorithms

Villa¹,

Salzo²,

Baldassarre³

et al. 2013

SIAM J. Optim.

179

224

View full text Add to dashboard Cite

Abstract. We propose a convergence analysis of accelerated forward-backward splitting methods for composite function minimization, when the proximity operator is not available in closed form, and can only be computed up to a certain precision. We prove that the 1/k 2 convergence rate for the function values can be achieved if the admissible errors are of a certain type and satisfy a sufficiently fast decay condition. Our analysis is based on the machinery of estimate sequences first introduced by Nesterov for the study of accelerated gradient descent algorithms. Furthermore, we give a global complexity analysis, taking into account the cost of computing admissible approximations of the proximal point. An experimental analysis is also presented.

show abstract

Section: Main Contributionsmentioning

confidence: 99%

Accelerated and Inexact Forward-Backward Algorithms

Villa¹,

Salzo²,

Baldassarre³

et al. 2013

SIAM J. Optim.

179

224

View full text Add to dashboard Cite

show abstract

“…The notion of an 'inexact oracle' was introduced in [25], and complexity estimates for primal, dual, and fast gradient methods applied to smooth convex functions with that inexact oracle are obtained. Further to this work, the same authors describe an intermediate gradient method that uses an inexact oracle [26]. That work is extended in [27] to handle the case of composite functions, where a stochastic inexact oracle is also introduced.…”

Section: Introductionmentioning

confidence: 99%

Inexact Coordinate Descent: Complexity and Preconditioning

Tappenden

Richtárik

Gondzio

2016

J Optim Theory Appl

View full text Add to dashboard Cite

One of the key steps at each iteration of a randomized block coordinate descent method consists in determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a particular subproblem is solved exactly. In this work, we relax this requirement and allow for the subproblem to be solved inexactly, leading to an inexact block coordinate descent method. Our approach incorporates the best known results for exact updates as a special case. Moreover, these theoretical guarantees are complemented by practical considerations: the use of iterative techniques to determine the update and the use of preconditioning for further acceleration.

show abstract

“…Some of these methods consider the case when gradient information is inaccurate. This error in the gradient computation may simply be bounded in the worst case (deterministically), see, for example, [11,20], or the error is random and the estimated gradient is accurate in expectation, as in stochastic gradient algorithms, see for example, [12,19,23,21]. These methods are typically applied in a convex setting and do not extend to nonconex cases.…”

Section: Introductionmentioning

confidence: 99%

“…In the nonlinear optimization setting, the complexity of various unconstrained methods has been derived under exact derivative information [7,8,17], and also under inexact information, where the errors are bounded in a deterministic fashion [3,6,11,14,20]. In all the cases of the deterministic inexact setting, traditional optimization algorithms such as line search, trust region or adaptive regularization algorithms are applied with little modification and work in practice as well as in theory, while the error is assumed to be bounded in some decaying manner at each iteration.…”

Section: Introductionmentioning

confidence: 99%

Global convergence rate analysis of unconstrained optimization methods based on probabilistic models

2017

View full text Add to dashboard Cite

We present global convergence rates for a line-search method which is based on random first-order models and directions whose quality is ensured only with certain probability. We show that in terms of the order of the accuracy, the evaluation complexity of such a method is the same as its counterparts that use deterministic accurate models; the use of probabilistic models only increases the complexity by a constant, which depends on the probability of the models being good. We particularize and improve these results in the convex and strongly convex case.We also analyze a probabilistic cubic regularization variant that allows approximate probabilistic second-order models and show improved complexity bounds compared to probabilistic first-order methods; again, as a function of the accuracy, the probabilistic cubic regularization bounds are of the same (optimal) order as for the deterministic case.

show abstract

First-order methods of smooth convex optimization with inexact oracle

Cited by 413 publications

References 12 publications

Accelerated and Inexact Forward-Backward Algorithms

Accelerated and Inexact Forward-Backward Algorithms

Inexact Coordinate Descent: Complexity and Preconditioning

Global convergence rate analysis of unconstrained optimization methods based on probabilistic models

Contact Info

Product

Resources

About