Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization

Taylor, Adrien; Hendrickx, Julien M.; Glineur, François

doi:10.1007/s10957-018-1298-1

Cited by 58 publications

(64 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because of this SSOS operation, our method is less dependent on the threshold (i.e., 20%) over the central surface of the k‐space than the method of, 58 who directly exploits the

{boldx}_{ℓ}^{LR}

images as sensitivity map information. Once the sensitivity maps were estimated, an accelerated proximal gradient method 59 was implemented to solve (11). The regularization parameter λ controls the tradeoff between data consistency and confidence in the sparsity prior, and this parameter was tuned manually over a discrete grid of values within the interval

[10^{- 7}; 10^{- 4}]

.…”

Section: Methodsmentioning

confidence: 99%

SPARKLING: variable‐density k‐space filling curves for accelerated T₂^*‐weighted MRI

Lazarus

Weiss

Chauffert

et al. 2019

Magnetic Resonance in Med

View full text Add to dashboard Cite

Purpose To present a new optimition‐driven design of optimal k‐space trajectories in the context of compressed sensing: Spreading Projection Algorithm for Rapid K‐space sampLING (SPARKLING). TheoryThe SPARKLING algorithm is a versatile method inspired from stippling techniques that automatically generates optimized sampling patterns compatible with MR hardware constraints on maximum gradient amplitude and slew rate. These non‐Cartesian sampling curves are designed to comply with key criteria for optimal sampling: a controlled distribution of samples (e.g., variable density) and a locally uniform k‐space coverage. MethodsEx vivo and in vivo prospective T2*‐weighted acquisitions were performed on a 7‐Tesla scanner using the SPARKLING trajectories for various setups and target densities. Our method was compared to radial and variable‐density spiral trajectories for high‐resolution imaging. ResultsCombining sampling efficiency with compressed sensing, the proposed sampling patterns allowed up to 20‐fold reductions in MR scan time (compared to fully sampled Cartesian acquisitions) for two‐dimensional T2*‐weighted imaging without deterioration of image quality, as demonstrated by our experimental results at 7 Tesla on in vivo human brains for a high in‐plane resolution of 390 μm. In comparison to existing non‐Cartesian sampling strategies, the proposed technique also yielded superior image quality. ConclusionsThe proposed optimization‐driven design of k‐space trajectories is a versatile framework that is able to enhance MR sampling performance in the context of compressed sensing.

show abstract

“…Because of this SSOS operation, our method is less dependent on the threshold (i.e., 20%) over the central surface of the k‐space than the method of, 58 who directly exploits the

{boldx}_{ℓ}^{LR}

[10^{- 7}; 10^{- 4}]

.…”

Section: Methodsmentioning

confidence: 99%

SPARKLING: variable‐density k‐space filling curves for accelerated T₂^*‐weighted MRI

Lazarus

Weiss

Chauffert

et al. 2019

Magnetic Resonance in Med

View full text Add to dashboard Cite

show abstract

“…Among them, a systematic approach to lower bounds (which focuses on quadratic cases) is presented in by Arjevani et al in [1], a systematic use of control theory (via integral quadratic constraints) for developing upper bounds is presented by Lessard et al in [26], and the performance estimation approach, which aims at finding worst-case bounds was originally developed in [13] (see also surveys in [11] and [47]). Those methodologies are mostly presented as tools for performing worst-cases analyses (see the numerous examples in [11,19,47,48,50,51]), however, such techniques were also recently used to develop new methods with improved worst-case complexities. Among others, such an approach was used in [13,22] to devise a fixed-step method that attains the best possible worst-case performance for smooth convex minimization [12], and later in [14] to obtain a variant of Kelley's cutting plane method with the best possible worst-case guarantee for non-smooth convex minimization.…”

Section: Links With Systematic and Computer-assisted Approaches To Womentioning

confidence: 99%

“…with τ ≥ 0 being as small as possible. Note that the presented analysis allows for more general initial conditions and performance measures (see discussions in [31,Section 4], and more specifically in [51,Section 4] in the context of performance estimation), however, for the sake of simplicity we do not pursue this direction in this work. We start the analysis of GFOM with the observation that, under the assumptions discussed above, the worst-case performance of GFOM is by definition the optimal value to the following performance estimation problem (PEP):…”

Section: Estimating the Worst-case Performance Of Gfommentioning

confidence: 99%

Efficient first-order methods for convex minimization: a constructive approach

Drori

Taylor

2019

Math. Program.

Self Cite

View full text Add to dashboard Cite

We describe a novel constructive technique for devising efficient first-order methods for a wide range of large-scale convex minimization settings, including smooth, non-smooth, and strongly convex minimization. The technique builds upon a certain variant of the conjugate gradient method to construct a family of methods such that a) all methods in the family share the same worst-case guarantee as the base conjugate gradient method, and b) the family includes a fixed-step first-order method. We demonstrate the effectiveness of the approach by deriving optimal methods for the smooth and non-smooth cases, including new methods that forego knowledge of the problem parameters at the cost of a one-dimensional line search per iteration, and a universal method for the union of these classes that requires a three-dimensional search per iteration. In the strongly convex case, we show how numerical tools can be used to perform the construction, and show that the resulting method offers an improved worst-case bound compared to Nesterov's celebrated fast gradient method. IntroductionConvex optimization plays a central role in many fields of applications, including optimal control, machine learning and signal processing. In particular, when a large number of variables are involved within a convex optimization problem, the use of first-order methods is more and more widespread due to their typically very attractive low computational cost per iteration. This low computational cost comes, however, at a price: first-order methods often suffer from potentially slow convergence speeds, making them appropriate mostly for obtaining low to medium accuracy solutions. Nevertheless, first-order methods remain the methods of choice in many applications and currently receive a lot of attention from the optimization community, which constantly aims at improving them.An effective and fruitful approach used for analyzing and comparing first-order methods is the study of their worst-case behavior through the black-box model. In this setting, methods are only allowed to gain information on the objective through an oracle, which provides the value and the gradient of the objective at selected points.

show abstract

“…There is a host of results in the literature using SDPs to analyze the convergence of first-order optimization algorithms [10,18,28,29]. The first among them is [10], in which Drori and Teboulle developed an SDP to derive analytical/numerical bounds on the worst-case performance of the unconstrained gradient method and its accelerated variant.…”

Section: Related Workmentioning

confidence: 99%

“…The first among them is [10], in which Drori and Teboulle developed an SDP to derive analytical/numerical bounds on the worst-case performance of the unconstrained gradient method and its accelerated variant. An extension of this framework to the proximal gradient method-for the case of strongly convex problems-has been recently proposed in [28]. These SDP formulations, despite being able to yield new performance bounds, are highly algorithm dependent.…”

Section: Related Workmentioning

confidence: 99%

Analysis of Optimization Algorithms via Integral Quadratic Constraints: Nonstrongly Convex Problems

Fazlyab¹,

Ribeiro²,

Morari³

et al. 2018

SIAM J. Optim.

134

View full text Add to dashboard Cite

In this paper, we develop a unified framework able to certify both exponential and subexponential convergence rates for a wide range of iterative first-order optimization algorithms. To this end, we construct a family of parameter-dependent nonquadratic Lyapunov functions that can generate convergence rates in addition to proving asymptotic convergence. Using Integral Quadratic Constraints (IQCs) from robust control theory, we propose a Linear Matrix Inequality (LMI) to guide the search for the parameters of the Lyapunov function in order to establish a rate bound. Based on this result, we formulate a Semidefinite Programming (SDP) whose solution yields the best convergence rate that can be certified by the class of Lyapunov functions under consideration. We illustrate the utility of our results by analyzing the gradient method, proximal algorithms and their accelerated variants for (strongly) convex problems. We also develop the continuous-time counterpart, whereby we analyze the gradient flow and the continuous-time limit of Nesterov's accelerated method.

show abstract

Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization

Cited by 58 publications

References 18 publications

SPARKLING: variable‐density k‐space filling curves for accelerated T₂^*‐weighted MRI

SPARKLING: variable‐density k‐space filling curves for accelerated T₂^*‐weighted MRI

Efficient first-order methods for convex minimization: a constructive approach

Analysis of Optimization Algorithms via Integral Quadratic Constraints: Nonstrongly Convex Problems

Contact Info

Product

Resources

About

Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization

Cited by 58 publications

References 18 publications

SPARKLING: variable‐density k‐space filling curves for accelerated T2*‐weighted MRI

SPARKLING: variable‐density k‐space filling curves for accelerated T2*‐weighted MRI

Efficient first-order methods for convex minimization: a constructive approach

Analysis of Optimization Algorithms via Integral Quadratic Constraints: Nonstrongly Convex Problems

Contact Info

Product

Resources

About

SPARKLING: variable‐density k‐space filling curves for accelerated T₂^*‐weighted MRI

SPARKLING: variable‐density k‐space filling curves for accelerated T₂^*‐weighted MRI