Smoothing augmented Lagrangian method for nonsmooth constrained optimization problems

Abstract

Abstract. In this paper, we propose a smoothing augmented Lagrangian method for finding a stationary point of a nonsmooth and nonconvex optimization problem. We show that any accumulation point of the iteration sequence generated by the algorithm is a stationary point provided that the penalty parameters are bounded. Furthermore, we show that a weak version of the generalized Mangasarian Fromovitz constraint qualification (GMFCQ) at the accumulation point is a sufficient condition for the boundedness of the penalty parameters. Since the weak GMFCQ may be strictly weaker than the GMFCQ, our algorithm is applicable for an optimization problem for which the GMFCQ does not hold. Numerical experiments show that the algorithm is efficient for finding stationary points of general nonsmooth and nonconvex optimization problems, including the bilevel program which will never satisfy the GMFCQ.

Relax-and-split method for nonconvex inverse problems

“…Our approach is closely related to Moreau-Yosida smoothing (see section 2 and [61]), which is used by many well-known algorithms, including those of [7,79], and [77]. If we partially minimize (7) with respect to w rather than with respect to x, we arrive at the problem min x h ν (Ax) + g(x), (11) with h ν analogous to the smoother discussed in [61].…”

Bilevel Polynomial Programs and Semidefinite Relaxation Methods

“…In [25,26], an algorithm using the branch and bound in combination with the exchange technique was proposed to find approximate global optimal solutions. Recently, the smoothing techniques were used to find stationary points of the valued function or the combined reformulation of simple bilevel programs [20,38,39,40].…”

“…Over the past two decades, many numerical algorithms were proposed for solving bilevel programs. However, most of them assume that the lower level program is convex, with few exceptions [20,25,26,31,38,39,40]. In [25,26], an algorithm using the branch and bound in combination with the exchange technique was proposed to find approximate global optimal solutions.…”

Relax-and-split method for nonsmooth nonconvex problems

“…Smoothing techniques are closely related to our approach; Moreau-Yosida smoothing (see Section 2) and related method of Nesterov (2005) are at the core of many well-known algorithms, including those of Becker et al (2011), Zhang (2011), andXu et al (2015). If we partially minimize (7) with respect to w rather than with respect to x, we arrive at the problem min…”