Learning Supervised PageRank with Gradient-Based and Gradient-Free Optimization Methods

Bogolubsky, Lev; Dvurechensky, Pavel; Gasnikov, Alexander; Gusev, Gleb; Nesterov, Yurii; Raigorodskii, Andrey; Tikhonov, Aleksey; Zhukovskii, Maxim

doi:10.48550/arxiv.1603.00717

Cited by 8 publications

(18 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We first show finite termination of the line search subroutine at each iteration, and establish a bound on the total number of function evaluations needed for its execution. The result is a generalization of the arguments in [7,44] for the case of relative smoothness in the nonconvex case. Lemma 5.12.…”

Section: Analysis Of Ahba(µ)mentioning

confidence: 69%

Generalized self-concordant analysis of Frank–Wolfe algorithms

et al. 2022

Self Cite

View full text Add to dashboard Cite

Projection-free optimization via different variants of the Frank–Wolfe method has become one of the cornerstones of large scale optimization for machine learning and computational statistics. Numerous applications within these fields involve the minimization of functions with self-concordance like properties. Such generalized self-concordant functions do not necessarily feature a Lipschitz continuous gradient, nor are they strongly convex, making them a challenging class of functions for first-order methods. Indeed, in a number of applications, such as inverse covariance estimation or distance-weighted discrimination problems in binary classification, the loss is given by a generalized self-concordant function having potentially unbounded curvature. For such problems projection-free minimization methods have no theoretical convergence guarantee. This paper closes this apparent gap in the literature by developing provably convergent Frank–Wolfe algorithms with standard $$\mathcal {O}(1/k)$$ O ( 1 / k ) convergence rate guarantees. Based on these new insights, we show how these sublinearly convergent methods can be accelerated to yield linearly convergent projection-free methods, by either relying on the availability of a local liner minimization oracle, or a suitable modification of the away-step Frank–Wolfe method.

show abstract

Section: Analysis Of Ahba(µ)mentioning

confidence: 69%

Generalized self-concordant analysis of Frank–Wolfe algorithms

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…The same type of dependence on the accuracy and the size of the problem can be seen for the working time (fig. 3): 10 Our experiments on MNIST data set show (see Figures 2, 3, 7) that in practice the bound is better. 11 Strictly speaking for the moment we can not verify all the details of the proof of estimate Õ(n 2 /ε).…”

Section: Numerical Illustrationmentioning

confidence: 82%

“…Further, we define Bregman divergence V [y](x) := d(x) − d(y) − ∇d(y), x − y . Next we define the inexact model of the objective function, which generalizes the inexact oracle of [19] (see also [24,10,28,35,60,62]). Definition 1.…”

Section: Gradient Methods With Inexact Model Of the Objectivementioning

confidence: 99%

See 1 more Smart Citation

Gradient Methods for Problems with Inexact Model of the Objective

Stonyakin

Dvinskikh

Dvurechensky

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We consider optimization methods for convex minimization problems under inexact information on the objective function. We introduce inexact model of the objective, which as a particular cases includes inexact oracle [19] and relative smoothness condition [43]. We analyze gradient method which uses this inexact model and obtain convergence rates for convex and strongly convex problems. To show potential applications of our general framework we consider three particular problems. The first one is clustering by electorial model introduced in [49]. The second one is approximating optimal transport distance, for which we propose a Proximal Sinkhorn algorithm. The third one is devoted to approximating optimal transport barycenter and we propose a Proximal Iterative Bregman Projections algorithm. We also illustrate the practical performance of our algorithms by numerical experiments.

show abstract

“…The literature on first-order methods [8,15,21] considers also gradient methods with inexact information, relaxing the model (2) to…”

Section: Introductionmentioning

confidence: 99%

Inexact Relative Smoothness and Strong Convexity for Optimization and Variational Inequalities by Inexact Model

Stonyakin¹,

Tyurin²,

Gasnikov³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many known methods as a special case, the list including accelerated gradient method, composite optimization methods, level-set methods, Bregman proximal methods. The idea of the framework is based on constructing an inexact model of the main problem component, i.e. objective function in optimization or operator in variational inequalities. Besides reproducing known results, our framework allows to construct new methods, which we illustrate by constructing a universal conditional gradient method and universal method for variational inequalities with composite structure. These method works for smooth and non-smooth problems with optimal complexity without a priori knowledge of the problem smoothness. As a particular case of our general framework, we introduce relative smoothness for operators and propose an algorithm for VIs with such operator. We also generalize our framework for relatively strongly convex objectives and strongly monotone variational inequalities.

show abstract

Learning Supervised PageRank with Gradient-Based and Gradient-Free Optimization Methods

Cited by 8 publications

References 11 publications

Generalized self-concordant analysis of Frank–Wolfe algorithms

Generalized self-concordant analysis of Frank–Wolfe algorithms

Gradient Methods for Problems with Inexact Model of the Objective

Inexact Relative Smoothness and Strong Convexity for Optimization and Variational Inequalities by Inexact Model

Contact Info

Product

Resources

About