Monotone Operators and the Proximal Point Algorithm

Abstract

We consider a Newton-CG augmented Lagrangian method for solving semidefinite programming (SDP) problems from the perspective of approximate semismooth Newton methods. In order to analyze the rate of convergence of our proposed method, we characterize the Lipschitz continuity of the corresponding solution mapping at the origin. For the inner problems, we show that the positive definiteness of the generalized Hessian of the objective function in these inner problems, a key property for ensuring the efficiency of using an inexact semismooth Newton-CG method to solve the inner problems, is equivalent to the constraint nondegeneracy of the corresponding dual problems. Numerical experiments on a variety of large-scale SDP problems with the matrix dimension n up to 4, 110 and the number of equality constraints m up to 2, 156, 544 show that the proposed method is very efficient. We are also able to solve the SDP problem fap36 (with n = 4, 110 and m = 1, 154, 467) in the Seventh DIMACS Implementation Challenge much more accurately than in previous attempts. -Yosida regularization approaches to solve SDP problems, and Jarre and Rendl [13] proposed an augmented primal-dual method for solving linear conic programs including SDP problems.In this paper, we study an augmented Lagrangian dual approach to solving largescale SDP problems with m large (say, up to a few million) but n moderate (say, up to 5, 000). Our approach is similar in spirit to those in [17] and [19], where the idea of augmented Lagrangian methods (or methods of multipliers in general) was heavily exploited. However, our point of view of employing the augmented Lagrangian methods is fundamentally different from [17] and [19] in solving both the outer and inner problems. It has long been known that the augmented Lagrangian method for convex problems is a gradient ascent method applied to the corresponding dual problems [30]. This inevitably leads to the impression that the augmented Lagrangian method for solving SDP problems may converge slowly for the outer iteration sequence {X k }. In spite of that, under mild conditions, a linear rate of convergence is available 1739 (superlinear convergence is also possible when σ k goes to infinity, which should be avoided in numerical implementations) [33]. However, recent studies conducted by Sun, Sun, and Zhang [37] and Chan and Sun [7] revealed that under the constraint nondegenerate conditions for (D) and (P ) (i.e., the dual and primal nondegeneracies in the IPM literature, e.g., [1]), respectively, the augmented Lagrangian method can be locally regarded as an approximate generalized Newton method applied to a semismooth equation. It is this connection that inspired us to investigate the augmented Lagrangian method for SDP problems. The approach of Jarre and Rendl [13] is to reformulate the problem as the minimization of a convex differentiable function in the primal-dual space. It is demonstrated in [13] that, numerically, the performance of using a nonlinear CG method to minimize this smooth convex function is qu...

Generic Optimality Conditions for Semialgebraic Convex Programs

“…For example, Rockafellar [33] observed that the classical proximal point method converges finitely on a polyhedral function; the same holds for functions with the "weak sharp minimum" property introduced by Ferris [12].…”

Convergence and Stability of a Regularization Method for Maximal Monotone Inclusions and Its Applications to Convex Optimization

“…[28]). The conditions under which the GPPM is known to converge strongly (see [52], [35], [7], [17], [20] and the references therein) are quite restrictive and mostly concern the data of ( ) [in contrast to those ensuring weak convergence which mostly concern the Bregman function whose selection can be done from a relatively large pool of known candidates -cf. [18]].…”

“…4.1 A question of interest in convex optimization concerns the strong convergence of the generalized proximal point method (GPPM for short) which emerged from the works of Martinet [43], [44], Rockafellar [52] and Censor and Zenios [21]. When applied to the consistent problem ( ) described in Subsection 3.1 the GPPM produces iterates according to the rule P { } …”

“…The origin of this question can be traced back to Rockafellar's work [52]. The relevance of the question emerges from the role of the proximal…”

“…In Section 4 we consider the question whether or under which conditions the generalized proximal point method for optimization which emerged from the works of Martinet [43], [44], Rockafellar [52] and Censor and Zenios [21] can be forced to converge strongly in infinite dimensional Banach spaces. The origin of this question can be traced back to Rockafellar's work [52].…”