An Augmented Lagrangian Method Exploiting an Active-Set Strategy and Second-Order Information

Abstract

In this paper, we consider nonlinear optimization problems with nonlinear equality constraints and bound constraints on the variables. For the solution of such problems, many augmented Lagrangian methods have been defined in the literature. Here, we propose to modify one of these algorithms, namely ALGENCAN by Andreani et al., in such a way to incorporate second-order information into the augmented Lagrangian framework, using an active-set strategy. We show that the overall algorithm has the same convergence properties as ALGENCAN and an asymptotic quadratic convergence rate under suitable assumptions. The numerical results confirm that the proposed algorithm is a viable alternative to ALGENCAN with greater robustness.

A decomposition method for lasso problems with zero-sum constraint

“…In the field of constrained optimization, several active-set techniques were proposed to identify the active (or binding) constraints, see, e.g., [4,6,10,11,13,16,17,22,23,34,36]. Active-set strategies were successfully used also to identify the zero variables in ℓ 1 -regularized problems [14,26,38,43,44] and in ℓ 1 -constrained problems [12].…”

“…Given a feasible point x k = 0 produced by the algorithm, in the sequel we define two possible strategies to compute π k in (13) and to update the variables. The first strategy uses the whole gradient ∇ϕ and is more accurate in the active-set estimate, while the second strategy only uses partial derivatives and is computationally more efficient.…”

“…and N k be defined as in (13), with π k computed as in (15) and µ(x k ) computed as in (14). Then, for any optimal solution x * of problem (1), there exists a neighborhood B(x * ) such that…”

“…This second strategy does not require to compute the whole gradient ∇ϕ. In (13) we simply set π k = π k−1 and then we update the variables in N k by means of an almost cyclic rule.…”

“…We have seen that, one the one hand, Strategy AC2CD does not need the expensive computation of the whole gradient ∇ϕ, but, on the other hand, we expect the active-set estimate used in Strategy MVP to be more accurate in a neighborhood of an optimal solution, according to Proposition 3.1. In order to balance computational efficiency and accuracy, we want to use Strategy MVP only when we judge it is worthwhile to compute a new gradient and a new π k in (13). This occurs when we observe no sufficient progress in the objective function.…”