Iteration-complexity of a proximal augmented Lagrangian method for solving nonconvex composite optimization problems with nonlinear convex constraints

Abstract

This paper proposes and analyzes a proximal augmented Lagrangian (NL-IAPIAL) method for solving smooth nonconvex composite optimization problems with nonlinear K-convex constraints, i.e., the constraints are convex with respect to the order given by a closed convex cone K. Each NL-IAPIAL iteration consists of inexactly solving a proximal augmented Lagrangian subproblem by an accelerated composite gradient (ACG) method followed by a Lagrange multiplier update. Under some mild assumptions, it is shown that NL-IAPIAL generates an approximate stationary solution of the aforementioned problem in at most O(log(1/ρ)/ρ 3 ) inner iterations, where ρ > 0 is a given tolerance.

An Accelerated Inexact Dampened Augmented Lagrangian Method for Linearly-Constrained Nonconvex Composite Optimization Problems

“…with an inexactness criterion (see (47)) previously considered by the authors in [9,11,12]. Third, it performs two kinds of iterations: (i) the ones indexed by k; and (ii) the ones that are performed by the ACG algorithm every time it is called in its Step 1.…”

“…In view of this fact, we say that a pair ([ẑ, p], v) is a (ρ, η)-approximate stationary point of (1) if it satisfies condition (6), which is clearly a relaxation of (11) for any (ρ, η) ∈ R 2 ++ .…”

“…This section examines the performance of the AIDAL method for solving problems of the form given in (1). It contains three subsections which each contain one of the following problem classes: (i) a class of linearly-constrained quadratic programming problems considered in [9]; (ii) the sparse principal component analysis (PCA) problem in [5]; and (iii) a class of linearly-constrained quadratic matrix problems considered in [10,11].…”

“…for every k ≥ 1; and (ii) the starting point given to the k th APG call is set to be x k−1 , which is the prox center for the k th prox subproblem. The second one, named IPL, is an implementation of the inexact proximal augmented Lagrangian method of [11,Section 5] where: (i) c k is doubled in its step 4 rather than quintupled; and (ii) σ = 0.3. The third one, named QP, is a practical modification of the quadratic penalty method of [9] in which: (i) each ACG subproblem in step 1 of the AIPP method is stopped when the condition u j + 2η j ≤ σ x 0 − x j + u j 2 holds; and (ii) it uses the parameters σ = 0.3 and c = max{1, M/ A 2 }.…”

“…Paper [22] presents O(ε −3 log ε −1 ) and O(ε −5/2 log ε −1 ) iteration complexities of an accelerated inexact PAL method for the general case and the case where (5) holds, respectively, and removes the requirement that the initial point be feasible. Finally, papers [11,21] present an O(ε −3 log ε −1 ) iteration complexity for the special case of (χ, θ) = (1, 0), which corresponds to a full multiplier update under the classical AL function.…”