Finding Second-Order Stationary Points in Nonconvex-Strongly-Concave Minimax Optimization

Abstract

We study the smooth minimax optimization problem of the form minx maxy f (x, y), where the objective function is strongly-concave in y but possibly nonconvex in x. This problem includes a lot of applications in machine learning such as regularized GAN, reinforcement learning and adversarial training. Most of existing theory related to gradient descent accent focus on establishing the convergence result for achieving the first-order stationary point of f (x, y) or primal function P (x) maxy f (x, y). In this paper, we design a new optimization method via cubic Newton iterations, which could find an O ε, κ 1.5 √ ρε -second-order stationary point of P (x) with O κ 1.5 √ ρε −1.5 second-order oracle calls and Õ κ 2 √ ρε −1.5 first-order oracle calls, where κ is the condition number and ρ is the Hessian smoothness coefficient of f (x, y). For high-dimensional problems, we propose an variant algorithm to avoid expensive cost form second-order oracle, which solves the cubic sub-problem inexactly via gradient descent and matrix Chebyshev expansion. This strategy still obtains desired approximate second-order stationary point with high probability but only requires Õ κ 1.5 ℓε −2 Hessian-vector oracle and Õ κ 2 √ ρε −1.5 first-order oracle calls. To the best of our knowledge, this is the first work considers non-asymptotic convergence behavior of finding second-order stationary point for minimax problem without convex-concave assumption.

Efficiently Escaping Saddle Points in Bilevel Optimization

“…However, all the previous works targeted finding stationary point of Φ(x). Very recently, [9,50] proposed cubic regularized GDA, a second-order algorithm that provably converges to a local minimum. [21] provided asymptotic results showing that GDA converges to local minimax point almost surely.…”

“…Remark 2.10 Compared with results of second-order methods escaping saddle points for minimax problem [9,50], our perturbed GDmax algorithm is purely first order, which means we do not need to compute Hessian-vector product. Moreover, algorithms in [9,50] require solving a nonconvex cubic sub-problem and multiple linear systems in each iteration. All these expensive computations are avoided in our perturbed GDmax algorithm, which makes it practical in real applications.…”

“…Recently, [9,50] proposed second-order algorithms for minimizing Φ(x), which is guaranteed to converge to local minimum. Second-order methods enjoy faster convergence rate than gradient methods, but require solving nonconvex subproblems in each iteration.…”