Metric selection in fast dual forward–backward splitting

Giselsson, Pontus; Boyd, Stephen

doi:10.1016/j.automatica.2015.09.010

Cited by 42 publications

(46 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As computation times become smaller and smaller, overheads due to the runtime environment get more and more relevant in the total CPU time. A tailored, low-level implementation of the same algorithm could significantly decrease the CPU times shown in Table I: this is also reported in [50], where a speedup of more than a factor 20 is observed using C code generation.…”

Section: ) Aircraft Controlsupporting

confidence: 56%

“…The dual problem has a condition number of 10 8 . To improve the convergence of the algorithms we therefore considered scaling the dual variables according to the Jacobi scaling, which consists of a diagonal change of variable (in the dual space) enforcing the (dual) Hessian to have diagonal elements equal to one (see also [50], [52] on the problem of preconditioning fast dual proximal gradient methods). Note that a diagonal change of variable in the dual space simply corresponds to a scaling of the equality constraints, when the problem is equivalently formulated as (P ).…”

Section: ) Aircraft Controlmentioning

confidence: 99%

See 1 more Smart Citation

Newton-type Alternating Minimization Algorithm for Convex Optimization

Stella

Themelis

Patrinos

2018

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

We propose NAMA (Newton-type Alternating Minimization Algorithm) for solving structured nonsmooth convex optimization problems where the sum of two functions is to be minimized, one being strongly convex and the other composed with a linear mapping. The proposed algorithm is a line-search method over a continuous, real-valued, exact penalty function for the corresponding dual problem, which is computed by evaluating the augmented Lagrangian at the primal points obtained by alternating minimizations. As a consequence, NAMA relies on exactly the same computations as the classical alternating minimization algorithm (AMA), also known as the dual proximal gradient method. Under standard assumptions the proposed algorithm possesses strong convergence properties, while under mild additional assumptions the asymptotic convergence is superlinear, provided that the search directions are chosen according to quasi-Newton formulas. Due to its simplicity, the proposed method is well suited for embedded applications and large-scale problems. Experiments show that using limited-memory directions in NAMA greatly improves the convergence speed over AMA and its accelerated variant. arXiv:1803.05256v1 [math.OC]

show abstract

Section: ) Aircraft Controlsupporting

confidence: 56%

Section: ) Aircraft Controlmentioning

confidence: 99%

Newton-type Alternating Minimization Algorithm for Convex Optimization

Stella

Themelis

Patrinos

2018

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

show abstract

“…This preconditioning is obtained by replacing the original minimization of f (x) + g(x) by minimization of f (Dq) + g(Dq), which gives rise to iterations involving the conjugate proximal maps D −1 F D (Dq), where F D is the proximal map for f as in (6) using the norm · (DD T ) −1 in place of the usual Euclidean metric. [15] includes some results about the rate of convergence as a function of D. In some cases, a larger value of ρ leads to faster convergence relative to ρ = 0.5. There are also results on convergence in the case that fixed ρ is replaced by a sequence of ρ k such that…”

Section: Anisotropic Preconditioned Mann Iteration For Nonexpansive Mapsmentioning

confidence: 99%

Plug-and-Play Unplugged: Optimization-Free Reconstruction Using Consensus Equilibrium

Buzzard¹,

Chan²,

Sreehari³

et al. 2018

SIAM J. Imaging Sci.

168

155

View full text Add to dashboard Cite

Regularized inversion methods for image reconstruction are used widely due to their tractability and their ability to combine complex physical sensor models with useful regularity criteria. Such methods motivated the recently developed Plug-and-Play prior method, which provides a framework to use advanced denoising algorithms as regularizers in inversion. However, the need to formulate regularized inversion as the solution to an optimization problem limits the expressiveness of possible regularity conditions and physical sensor models.In this paper, we introduce the idea of Consensus Equilibrium (CE), which generalizes regularized inversion to include a much wider variety of both forward (or data fidelity) components and prior (or regularity) components without the need for either to be expressed using a cost function. Consensus equilibrium is based on the solution of a set of equilibrium equations that balance data fit and regularity. In this framework, the problem of MAP estimation in regularized inversion is replaced by the problem of solving these equilibrium equations, which can be approached in multiple ways.The key contribution of CE is to provide a novel framework for fusing multiple heterogeneous models of physical sensors or models learned from data. We describe the derivation of the CE equations and prove that the solution of the CE equations generalizes the standard MAP estimate under appropriate circumstances.We also discuss algorithms for solving the CE equations, including a version of the Douglas-Rachford (DR)/ADMM algorithm with a novel form of preconditioning and Newton's method, both standard form and a Jacobian-free form using Krylov subspaces. We give several examples to illustrate the idea of consensus equilibrium and the convergence properties of these algorithms and demonstrate this method on some toy problems and on a denoising example in which we use an array of convolutional neural network denoisers, none of which is tuned to match the noise level in a noisy image but which in consensus can achieve a better result than any of them individually.

show abstract

“…First-order methods are known to be sensitive to scaling and preconditioning can remarkably improve their convergence rate. Various preconditioning method such as [42], [43] have been proposed in the literature. Here, we employ a simple diagonal preconditioning which consists in computing a diagonal matrixH D with positive diagonal entries which approximates the dual Hessian H D and useH −1/2 D to scale the dual vector [44, 2.3.1].…”

Section: Preconditioning and Choice Of λmentioning

confidence: 99%

GPU-Accelerated Stochastic Predictive Control of Drinking Water Networks

Sampathirao

Sopasakis

Bemporad

et al. 2018

IEEE Trans. Contr. Syst. Technol.

View full text Add to dashboard Cite

Abstract-Despite the proven advantages of scenario-based stochastic model predictive control for the operational control of water networks, its applicability is limited by its considerable computational footprint. In this paper we fully exploit the structure of these problems and solve them using a proximal gradient algorithm parallelizing the involved operations. The proposed methodology is applied and validated on a case study: the water network of the city of Barcelona.

show abstract

Metric selection in fast dual forward–backward splitting

Cited by 42 publications

References 30 publications

Newton-type Alternating Minimization Algorithm for Convex Optimization

Newton-type Alternating Minimization Algorithm for Convex Optimization

Plug-and-Play Unplugged: Optimization-Free Reconstruction Using Consensus Equilibrium

GPU-Accelerated Stochastic Predictive Control of Drinking Water Networks

Contact Info

Product

Resources

About