A family of functional inequalities: Łojasiewicz inequalities and displacement convex functions

Blanchet, Adrien; Bolte, Jérôme

doi:10.1016/j.jfa.2018.06.014

Cited by 17 publications

(35 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, our local analysis includes a functional and a gradient Łojasiewicz inequality of order 2 in Wasserstein space. Such inequalities were studied in [34,7] for displacement convex functions, which does not cover our setting.…”

Section: Related Techniquesmentioning

confidence: 99%

Sparse optimization on measures with over-parameterized gradient descent

Chizat

2021

Math. Program.

View full text Add to dashboard Cite

Minimizing a convex function of a measure with a sparsity-inducing penalty is a typical problem arising, e.g., in sparse spikes deconvolution or two-layer neural networks training. We show that this problem can be solved by discretizing the measure and running nonconvex gradient descent on the positions and weights of the particles. For measures on a d-dimensional manifold and under some non-degeneracy assumptions, this leads to a global optimization algorithm with a complexity scaling as log(1/ ) in the desired accuracy , instead of −d for convex methods. The key theoretical tools are a local convergence analysis in Wasserstein space and an analysis of a perturbed mirror descent in the space of measures. Our bounds involve quantities that are exponential in d which is unavoidable under our assumptions.

show abstract

Section: Related Techniquesmentioning

confidence: 99%

Sparse optimization on measures with over-parameterized gradient descent

Chizat

2021

Math. Program.

View full text Add to dashboard Cite

show abstract

“…• linear rates if p = 2. For more examples, related notions and references, we refer the interested reader to [16,5,20]. Corollary 4.1 highlights the fact that the behavior of the thresholding gradient method essentially depends on the conditioning of f on finitely supported subspaces.…”

Section: Strong Convergence and Ratesmentioning

confidence: 99%

Thresholding gradient methods in Hilbert spaces: support identification and linear convergence

Garrigos¹,

Rosasco

Villa

2020

ESAIM: COCV

View full text Add to dashboard Cite

We study 1 regularized least squares optimization problem in a separable Hilbert space. We show that the iterative soft-thresholding algorithm (ISTA) converges linearly, without making any assumption on the linear operator into play or on the problem. The result is obtained combining two key concepts: the notion of extended support, a finite set containing the support, and the notion of conditioning over finite dimensional sets. We prove that ISTA identifies the solution extended support after a finite number of iterations, and we derive linear convergence from the conditioning property, which is always satisfied for 1 regularized least squares problems. Our analysis extends to the the entire class of thresholding gradient algorithms, for which we provide a conceptually new proof of strong convergence, as well as convergence rates. 1 arXiv:1712.00357v1 [math.OC] 1 Dec 2017 subspace, ensuring automatically strong convergence. The key argument in our analysis is that after a finite number of iterations, the iterates identify the so called extended support of their limit. This set coincides with the active constraints at the solution of the dual problem, and reduces to the support, if some qualification condition is satisfied. Going further, we tackle the question of convergence rates, providing a unifying treatment of finite and infinite dimensional settings. In finite dimensions, it is clear that if A is injective, then f becomes a strongly convex function, which guarantees a linear convergence rate. In [22], it is shown, still in a finite dimensional setting, that the linear rates hold just assuming A to be injective on the extended support of the problem. This result is generalized in [8] to a Hilbert space setting, assuming A to be injective on any subspace of finite support. Linear convergence is also obtained by assuming the limit solution to satisfy some nondegeneracy condition [8,26]. In fact, it was shown recently in [6] that, in finite dimension, no assumption at all is needed to guarantee linear rates. Using a key result in [25], the function f was shown to be 2-conditioned on its sublevel sets, and 2-conditioning is sufficient for linear rates [2]. Our identification result, mentioned above, allows to easily bridge the gap between the finite and infinite dimensional settings. Indeed, we show that in any separable Hilbert space, linear rates of convergence always hold for the soft-thresholding gradient algorithm under no further assumptions. Once again, the key argument to obtain linear rates is the fact that the iterates generated by the algorithm identify, in finite time, a set on which we know the function to have a favorable geometry.The paper is organized as follows. In Section 2 we describe our setting and introduce the thresholding gradient method. We introduce the notion of extended support in Section 3, in which we show that the thresholding gradient algorithm identifies this extended support after a finite number of iterations (Theorem 3.9). In Section 4 we present some consequences of this res...

show abstract

“…They adapted Kurdyka's notion of a talweg curve to the Hilbert space framework and characterized the validity of a K L-inequality (1.6) with the existence of a talweg. In addition, first formulations of K L-inequality (1.6) in a metric space framework were also given in [17] (see also the recent work [15]).…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we introduce local and global K L-inequalities (1.6) for proper functionals E : M → (−∞, +∞] defined on a metric space (M, d) (see Definition 3.2 in Section 3.1). Our definition here is slightly different to the one in [17,15], but consistent with one in the Hilbert space framework given, for instance, by [29]. This enables us to provide new fine tools for determining the trend to equilibrium in both the entropy sense (in Section 2.5) and the metric sense (in Section 3.5) of gradient flows in M. More precisely, we show in Theorem 3.5 that if E is bounded from below and satisfies a K L-inequality (1.6) on a set U then every gradient flow v of E satisfying v([t 0 , +∞)) ⊆ U for some t 0 ≥ 0 has finite length.…”

Section: Introductionmentioning

confidence: 99%

“…Then, as an application of this, they establish the equivalence between Talagrand's inequality (1.11) and the logarithmic Sobolev inequality (1.10) for the Boltzmann H-functional (1.9) on R N for V = 0. The method in [15] is based on a similar idea to ours (cf Theorem 3.32), but our results presented in Section 3.8 are concerned with a much general framework. In our forthcoming paper, we show how these results help us to study the geometry of metric measure length space with a Ricci-curvature bound in the sense of Sturm [56,57] and Lott-Villani [48].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation