Accelerated Primal-Dual Gradient Descent with Linesearch for Convex, Nonconvex, and Nonsmooth Optimization Problems

Guminov, Sergey; Nesterov, Yurii; Dvurechensky, Pavel; Gasnikov, Alexander

doi:10.1134/s1064562419020042

Cited by 28 publications

(13 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the star network, we can compare the complexity of Mirror-Prox with the complexity of the IBP running in O n 2 /ε 2 time per node [19]. Distributed Mirror-Prox has better dependence on ε, namely 1/ε, as well as the accelerated IBP with O n 2 √ n/ε complexity per node of [15]. However, as any regularization-based method, the (accelerated) IBP has a strong limitation regarding the regularization parameter: numerical unstability at small regularization parameter.…”

Section: Application Of Distributed Mirror-proxmentioning

confidence: 99%

Decentralized Distributed Optimization for Saddle Point Problems

Rogozin¹,

Beznosikov²,

Dvinskikh³

et al. 2021

Preprint

View full text Add to dashboard Cite

We consider distributed convex-concave saddle point problems over arbitrary connected undirected networks and propose a decentralized distributed algorithm for their solution. The local functions distributed across the nodes are assumed to have global and local groups of variables. For the proposed algorithm we prove non-asymptotic convergence rate estimates with explicit dependence on the network characteristics. To supplement the convergence rate analysis, we propose lower bounds for strongly-convex-strongly-concave and convex-concave saddle-point problems over arbitrary connected undirected networks. We illustrate the considered problem setting by a particular application to distributed calculation of non-regularized Wasserstein barycenters.

show abstract

Section: Application Of Distributed Mirror-proxmentioning

confidence: 99%

Decentralized Distributed Optimization for Saddle Point Problems

Rogozin¹,

Beznosikov²,

Dvinskikh³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In this section similarly with the concept of (δ, L, µ, m, V )-model in optimization we consider inexact model for VI with a stronger version of monotonicity condition (34).…”

Section: Inexact Model For Strongly Monotone VImentioning

confidence: 99%

“…We believe that our model is flexible enough to be extended for problems with primal-dual structure 2 [49,51,54], e.g. for problems with linear constraints [2,12,34,58]; for random block-coordinate descent [25]; for tensor methods [30,55]; for distributed optimization setting [17,18,64,68]; and adaptive stochastic optimization [36,61].…”

Section: Introductionmentioning

confidence: 99%

Inexact Relative Smoothness and Strong Convexity for Optimization and Variational Inequalities by Inexact Model

Stonyakin¹,

Tyurin²,

Gasnikov³

et al. 2020

Preprint

View full text Add to dashboard Cite

In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many known methods as a special case, the list including accelerated gradient method, composite optimization methods, level-set methods, Bregman proximal methods. The idea of the framework is based on constructing an inexact model of the main problem component, i.e. objective function in optimization or operator in variational inequalities. Besides reproducing known results, our framework allows to construct new methods, which we illustrate by constructing a universal conditional gradient method and universal method for variational inequalities with composite structure. These method works for smooth and non-smooth problems with optimal complexity without a priori knowledge of the problem smoothness. As a particular case of our general framework, we introduce relative smoothness for operators and propose an algorithm for VIs with such operator. We also generalize our framework for relatively strongly convex objectives and strongly monotone variational inequalities.

show abstract

“…Then we analyze the convergence rate of our primal-dual version of the Algorithm 1. Note that the primal-dual analysis of the existing accelerated methods [66,2,7,12,23,24,25,26,33,50] does not apply since the dual problem is a stochastic optimization problem and we use additional randomization. Algorithm 6.1 of [18] applied to the dual problem (22) with stochastic inexact oracle ∇ Φ(λ, ξ, ξ) is listed as Algorithm C3.…”

Section: Proof Of Theoremmentioning

confidence: 99%

Distributed Optimization with Quantization for Computing Wasserstein Barycenters

Krawtschenko¹,

Uribe²,

Gasnikov³

et al. 2020

Preprint

View full text Add to dashboard Cite

We study the problem of the decentralized computation of entropy-regularized semidiscrete Wasserstein barycenters over a network. Building upon recent primal-dual approaches, we propose a sampling gradient quantization scheme that allows efficient communication and computation of approximate barycenters where the factor distributions are stored distributedly on arbitrary networks. The communication and algorithmic complexity of the proposed algorithm are shown, with explicit dependency on the size of the support, the number of distributions, and the desired accuracy. Numerical results validate our algorithmic analysis.

show abstract

Accelerated Primal-Dual Gradient Descent with Linesearch for Convex, Nonconvex, and Nonsmooth Optimization Problems

Cited by 28 publications

References 6 publications

Decentralized Distributed Optimization for Saddle Point Problems

Decentralized Distributed Optimization for Saddle Point Problems

Inexact Relative Smoothness and Strong Convexity for Optimization and Variational Inequalities by Inexact Model

Distributed Optimization with Quantization for Computing Wasserstein Barycenters

Contact Info

Product

Resources

About