High-order evaluation complexity for convexly-constrained optimization with non-Lipschitzian group sparsity terms

Abstract

This paper studies high-order evaluation complexity for partially separable convexlyconstrained optimization involving non-Lipschitzian group sparsity terms in a nonconvex objective function. We propose a partially separable adaptive regularization algorithm using a p-th order Taylor model and show that the algorithm can produce an (ǫ, δ)approximate q-th-order stationary point at most O(ǫ −(p+1)/(p−q+1) ) evaluations of the objective function and its first p derivatives (whenever they exist). Our model uses the underlying rotational symmetry of the Euclidean norm function to build a Lipschitzian approximation for the non-Lipschitzian group sparsity terms, which are defined by the group ℓ 2 -ℓ a norm with a ∈ (0, 1). The new result shows that the partially-separable structure and non-Lipschitzian group sparsity terms in the objective function may not affect the worst-case evaluation complexity order.

Tensor methods for finding approximate stationary points of convex functions

“…After [2], several high-order methods have been proposed and analysed for nonconvex optimization (see, e.g. [9][10][11]23]), resulting even in worst-case complexity bounds for the number of iterations that p-order methods need to generate approximate qth order stationary points [7,8].…”

“…If ν is unknown, by (11) we set α = 1 in Algorithm 1. The resulting algorithm is a universal scheme that can be viewed as a generalization of the universal second-order method (6.10) in [16].…”

“…Assume that (17) is true for some t, 0 ≤ t ≤ T − 1. If ν is known, then by (11) we have α = ν. Thus, it follows from H1 and Lemma A.2 in [18] that the final value of 2 i t H t cannot be bigger than…”

An Adaptive High Order Method for Finding Third-Order Critical Points of Nonconvex Optimization

“…

It is well known that finding a global optimum is extremely challenging for nonconvex optimization. There are some recent efforts [1,[12][13][14] regarding the optimization methods for computing higherorder critical points, which can exclude the so-called degenerate saddle points and reach a solution with better quality. Desipte theoretical development in [1,[12][13][14], the corresponding numerical experiments are missing.

…”

“…There are some recent efforts [1,[12][13][14] regarding the optimization methods for computing higherorder critical points, which can exclude the so-called degenerate saddle points and reach a solution with better quality. Desipte theoretical development in [1,[12][13][14], the corresponding numerical experiments are missing. In this paper, we propose an implementable higher-order method, named adaptive high order method (AHOM), that aims to find the third-order critical points.…”