A primal-dual flow for affine constrained convex optimization

Abstract

We introduce a novel primal-dual flow for affine constrained convex optimization problems. As a modification of the standard saddle-point system, our primal-dual flow is proved to possess the exponential decay property, in terms of a tailored Lyapunov function. Then two primal-dual methods are obtained from numerical discretizations of the continuous model, and global nonergodic linear convergence rate is established via a discrete Lyapunov function. Instead of solving the subproblem of the primal variable, we apply the semi-smooth Newton iteration to the subproblem with respect to the multiplier, provided that there are some additional properties such as semi-smoothness and sparsity. Especially, numerical tests on the linearly constrained l 1 -l 2 minimization and the total-variation based image denoising model have been provided.

Faster augmented Lagrangian method for solving convex optimization problems with linear constraints

“…Subsequently, He et al [34] proposed an inertial ALM with a non-ergodic convergence rate of O(1/k p ) (p ≥ 1) for both function values and the feasibility measure. However, when tackling (P2), despite the easily solvable subproblems of ADMM-type algorithms, the subproblems presented in inertial ALM-type algorithms by [32,33,37] may not be as straightforward. This is the reason why the convergence rate surpasses O(1/k) without the need for additional conditions such as strong convexity or Lipschitz continuity of the gradient.…”

“…Bot ¸et al [35] and Hulett and Nguyen [36] also analyzed the case when f is Lipschitz continuous with convergent iterative sequences using the second-order dynamical system. When addressing (P2), despite the easily solvable subproblems of ADMMtype algorithms, the subproblems of inertial ALM-type algorithms given in [32,33,37] may not be as easily solvable. This is why the convergence rate is faster than O(1/k) without additional conditions such as strong convexity or Lipschitz continuity of the gradient.…”