On the Complexity of an Augmented Lagrangian Method for Nonconvex Optimization

Abstract

In this paper we study the worst-case complexity of an inexact Augmented Lagrangian method for nonconvex constrained problems. Assuming that the penalty parameters are bounded, we prove a complexity bound of O(| log(ǫ)|) iterations for the referred algorithm generate an ǫ-approximate KKT point, for ǫ ∈ (0, 1). When the penalty parameters are unbounded, we prove an iteration complexity bound of O ǫ −2/(α−1) , where α > 1 controls the rate of increase of the penalty parameters. For linearly constrained problems, these bounds yield to evaluation complexity, respectively, when suitable p-order methods (p ≥ 2) are used to approximately solve the unconstrained subproblems at each iteration of our Augmented Lagrangian scheme.

A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees

“…Turning to (27), we note first that from termination conditions of Algorithm 2 that rk ≤ ζr g . Thus, using (28)…”

“…However, these papers do not account for the cost of solving the subproblem at each iteration, noting either that this subproblem may be NP-hard, or suggesting that a simple first-order, gradient-based method can solve it reliably. Many other methods have been proposed for constrained optimization which have good worst-case iteration complexity results, such as two-phase methods [4,12,24], an interior-point method [31], and augmented Lagrangian methods [7,28,38].…”

Complexity of Proximal augmented Lagrangian for nonconvex optimization with nonlinear equality constraints

“…There are few results in the literature on outer iteration complexity in the nonconvex setting. Some quite recent results appear in [13,10]. In [13], the authors apply a general version of augmented Lagrangian to nonconvex optimization with both equality and inequality constraints.…”

“…Some quite recent results appear in [13,10]. In [13], the authors apply a general version of augmented Lagrangian to nonconvex optimization with both equality and inequality constraints. With an aggressive updating rule for the penalty parameter, they show that the algorithm obtains an approximate KKT point (whose exact definition is complicated, but similar to our definition of ǫ-1o optimality when only equality constraints are present) within O(ǫ −2/(α−1) ) outer-loop iterations, where α > 1 is an algorithmic parameter.…”

Complexity and performance of an Augmented Lagrangian algorithm

“…An Exact Penalty algorithm for constrained optimization with complexity results was introduced in [24]. Grapiglia and Yuan [38] analyzed the complexity of an Augmented Lagrangian algorithm for inequality constraints based on the approach of Sun and Yuan [46] and assuming that a feasible initial point is available.…”