Forward-Backward Splitting with Bregman Distances

Nguyen, Quang Van

doi:10.1007/s10013-016-0238-3

Cited by 54 publications

(52 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many other first order proximal methods for Bregman divergences exists. The most simple one is the proximal point algorithm [31], but most proximal splitting schemes have been extended to this setting, such as for instance ADMM [78], primal-dual schemes [23] and forward-backward [54].…”

Section: Introductionmentioning

confidence: 99%

Scaling algorithms for unbalanced optimal transport problems

et al. 2018

View full text Add to dashboard Cite

This article introduces a new class of fast algorithms to approximate variational problems involving unbalanced optimal transport. While classical optimal transport considers only normalized probability distributions, it is important for many applications to be able to compute some sort of relaxed transportation between arbitrary positive measures. A generic class of such "unbalanced" optimal transport problems has been recently proposed by several authors. In this paper, we show how to extend the, now classical, entropic regularization scheme to these unbalanced problems. This gives rise to fast, highly parallelizable algorithms that operate by performing only diagonal scaling (i.e. pointwise multiplications) of the transportation couplings. They are generalizations of the celebrated Sinkhorn algorithm. We show how these methods can be used to solve unbalanced transport, unbalanced gradient flows, and to compute unbalanced barycenters. We showcase applications to 2-D shape modification, color transfer, and growth models. arXiv:1607.05816v2 [math.OC]

show abstract

Section: Introductionmentioning

confidence: 99%

Scaling algorithms for unbalanced optimal transport problems

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Zhou et al [17] discusses a unified proof of Mirror Descent and the Proximal Point Algorithm under a similar assumption of relative smoothness. Nguyen [15] develops similar ideas on analyzing Mirror Descent in a Banach space. A more detailed discussion comparing these related works is also presented in [8].…”

Section: Introductionmentioning

confidence: 99%

“Relative Continuity” for Non-Lipschitz Nonsmooth Convex Optimization Using Stochastic (or Deterministic) Mirror Descent

2019

INFORMS Journal on Optimization

View full text Add to dashboard Cite

The usual approach to developing and analyzing first-order methods for non-smooth (stochastic or deterministic) convex optimization assumes that the objective function is uniformly Lipschitz continuous with parameter M f . However, in many settings the non-differentiable convex function f is not uniformly Lipschitz continuous -for example (i) the classical support vector machine (SVM) problem, (ii) the problem of minimizing the maximum of convex quadratic functions, and even (iii) the univariate setting with f (x) := max{0, x} + x 2 . Herein we develop a notion of "relative continuity" that is determined relative to a user-specified "reference function" h (that should be computationally tractable for algorithms), and we show that many non-differentiable convex functions are relatively continuous with respect to a correspondingly fairly-simple reference function h. We also similarly develop a notion of "relative stochastic continuity" for the stochastic setting. We analyze two standard algorithms -the (deterministic) mirror descent algorithm and the stochastic mirror descent algorithm -for solving optimization problems in these new settings, providing the first computational guarantees for instances where the objective function is not uniformly Lipschitz continuous. This paper is a companion paper for non-differentiable convex optimization to the recent paper by Lu, Freund, and Nesterov, which developed analogous results for differentiable convex optimization. *

show abstract

“…In the 2008 preprint [44], the space X is a real normed space which is probably either finite-dimensional (there are many similarities to [45] where X is finite-dimensional) or at least a reflexive Banach space, since otherwise it is not clear why the iterations in various places, for instance in [44,Equations (8), (12), (17), (28), (31), (45), (47), (48)], are well defined. The paper [44] is the only relevant paper of which we are aware, where there are subsets (closed and convex) S k on which the iterations depend, but they are not almost arbitrary as in our method (see Algorithm 5.5 and Algorithm 5.6): rather, they are either the whole space or they have a certain complicated form (which seems to be inspired by [28, p. 240]) in order to make sure that each of them contains an optimal solution; see [44,Equations (16), (46)]; on the other hand, they are not assumed to satisfy ∪ ∞ k=1 S k = C and S k ⊆ S k+1 for all k as in our case (Assumption 5.2 below): actually, at least in [44,p.…”

Section: Further Comparison To the Literaturementioning

confidence: 99%

“…It follows from the equality F (x) = f (x) + g(x) for every x ∈ S, from (28), (30), (31), from (11) with z instead of x, from Lemma 8.4, and from (5), that for all x ∈ S, we have…”

Section: Proofsmentioning

confidence: 99%

A Telescopic Bregmanian Proximal Gradient Method Without the Global Lipschitz Continuity Assumption

Reem

Reich

Pierro

2019

J Optim Theory Appl

View full text Add to dashboard Cite

The problem of minimization of the sum of two convex functions has various theoretical and real-world applications. One of the popular methods for solving this problem is the proximal gradient method (proximal forward-backward algorithm). A very common assumption in the use of this method is that the gradient of the smooth term is globally Lipschitz continuous. However, this assumption is not always satisfied in practice, thus casting a limitation on the method. In this paper, we discuss, in a wide class of finite and infinite-dimensional spaces, a new variant of the proximal gradient method which does not impose the above-mentioned global Lipschitz continuity assumption. A key contribution of the method is the dependence of the iterative steps on a certain telescopic decomposition of the constraint set into subsets. Moreover, we use a Bregman divergence in the proximal forward-backward operation. Under certain practical conditions, a non-asymptotic rate of convergence (that is, in the function values) is established, as well as the weak convergence of the whole sequence to a minimizer. We also obtain a few auxiliary results of independent interest.

show abstract

Forward-Backward Splitting with Bregman Distances

Cited by 54 publications

References 22 publications

Scaling algorithms for unbalanced optimal transport problems

Scaling algorithms for unbalanced optimal transport problems

“Relative Continuity” for Non-Lipschitz Nonsmooth Convex Optimization Using Stochastic (or Deterministic) Mirror Descent

A Telescopic Bregmanian Proximal Gradient Method Without the Global Lipschitz Continuity Assumption

Contact Info

Product

Resources

About