Iteration-complexity of an inexact proximal accelerated augmented Lagrangian method for solving linearly constrained smooth nonconvex composite optimization problems

Abstract

This paper proposes and establishes the iteration-complexity of an inexact proximal accelerated augmented Lagrangian (IPAAL) method for solving linearly constrained smooth nonconvex composite optimization problems. Each IPAAL iteration consists of inexactly solving a proximal augmented Lagrangian subproblem by an accelerated composite gradient (ACG) method followed by a suitable Lagrange multiplier update. It is shown that IPAAL generates an approximate stationary solution in at most O(log(1/ρ)/ρ 3 ) ACG iterations, where ρ > 0 is the given tolerance. It is also shown that the previous complexity bound can be sharpened to O(log(1/ρ)/ρ 2.5 ) under additional mildly stronger assumptions. The above bounds are derived assuming that the initial point is neither feasible nor the domain of the composite term of the objective function is bounded. Some preliminary numerical results are presented to illustrate the performance of the IPAAL method.

An Accelerated Inexact Dampened Augmented Lagrangian Method for Linearly-Constrained Nonconvex Composite Optimization Problems

“…Similar to the analyses in [15,18], the analysis of the AIDAL method strongly makes use of assumption (A3) and the assumption that D h < ∞ to obtain its competitive O(ε −5/2 log ε −1 ) iteration complexity. However, we conjecture that the these two assumptions may be removed using the more complicated analysis in [22] to obtain a slightly worse O(ε −3 log ε −1 ) iteration complexity (like in [22]).…”

“…Paper [6] presents an O(ε −4 ) iteration complexity of an unaccelerated PAL method under the strong assumption that the initial point z 0 is feasible, i.e., Az 0 = b, and where θ ∈ (0, 1] and χ = 1. Paper [22] presents O(ε −3 log ε −1 ) and O(ε −5/2 log ε −1 ) iteration complexities of an accelerated inexact PAL method for the general case and the case where (5) holds, respectively, and removes the requirement that the initial point be feasible. Finally, papers [11,21] present an O(ε −3 log ε −1 ) iteration complexity for the special case of (χ, θ) = (1, 0), which corresponds to a full multiplier update under the classical AL function.…”

“…We now emphasize how the proposed AIDAL method improves on other state-ofthe-art AL-based works. First, in contrast to the O(ε −5/2 log ε −1 ) complexity PAL method in [22] where λ → 0 as θ → 0, it chooses λ in update (3) independent of θ. Moreover, as θ → 0 the AIDAL method becomes a penalty method with stepsize λ (see Subsection 2.2) while the method in [22] becomes a PAL method with arbitrarily small stepsize.…”

“…First, in contrast to the O(ε −5/2 log ε −1 ) complexity PAL method in [22] where λ → 0 as θ → 0, it chooses λ in update (3) independent of θ. Moreover, as θ → 0 the AIDAL method becomes a penalty method with stepsize λ (see Subsection 2.2) while the method in [22] becomes a PAL method with arbitrarily small stepsize. Second, it improves on the O(ε −5/2 log ε −1 ) AL-based method in [15] in two significant ways: (i) it performs the multiplier update (4) after every inexact prox update as opposed to only when the penalty parameter is updated; and (ii) it chooses a constant under-relaxation parameter χ for the update (4) as opposed to in [15] which chooses a under-relaxation parameter that (linearly) tends to zero as the number of penalty parameter updates increases.…”

Iteration-complexity of a proximal augmented Lagrangian method for solving nonconvex composite optimization problems with nonlinear convex constraints

“…Paper [7] introduces a perturbed θ-AL function, which agrees with the classical one in (2) when θ = 0, and studies a corresponding unaccelerated PAL method whose iteration complexity is O(η −4 + ρ−4 ) under the strong condition that the initial point z 0 is feasible. Paper [22] analyzes the iteration-complexity of an inexact proximal accelerated PAL method based on the aforementioned perturbed AL function and shows, regardless of whether the initial point is feasible, that a solution to (1) is obtained in O(η −1 ρ−2 log η−1 ) ACG iterations and that the latter bound can be improved to O(η −1/2 ρ−2 log η−1 ) under an additional Slater-like assumption. Both papers [7,22] assume that θ ∈ (0, 1], and hence, their analyses do not apply to the classical PAL method.…”