Implicit regularity and linear convergence rates for the generalized trust-region subproblem

Abstract

In this paper we develop efficient first-order algorithms for the generalized trust-region subproblem (GTRS), which has applications in signal processing, compressed sensing, and engineering. Although the GTRS, as stated, is nonlinear and nonconvex, it is well-known that objective value exactness holds for its SDP relaxation under a Slater condition. While polynomial-time SDP-based algorithms exist for the GTRS, their relatively large computational complexity has motivated and spurred the development of custom approaches for solving the GTRS. In particular, recent work in this direction has developed first-order methods for the GTRS whose running times are linear in the sparsity (the number of nonzero entries) of the input data. In contrast to these algorithms, in this paper we develop algorithms for computing -approximate solutions to the GTRS whose running times are linear in both the input sparsity and the precision log(1/ ) whenever a regularity parameter is positive. We complement our theoretical guarantees with numerical experiments comparing our approach against algorithms from the literature. Our numerical experiments highlight that our new algorithms significantly outperform prior state-of-the-art algorithms on sparse large-scale instances.

Solving Stackelberg Prediction Game with Least Squares Loss via Spherically Constrained Least Squares Reformulation

“…Problem ( 12) is closed related to the well-known trust region subproblem (TRS), where the sphere constraint is replaced by a unit ball constraint. There exist various methods for solving (12) from the literature on TRS (Moré & Sorensen, 1983;Gould et al, 1999;Conn et al, 2000;Hazan & Koren, 2016;Ho-Nguyen & Kilinc-Karzan, 2017;Zhang et al, 2017;Carmon & Duchi, 2018), or the generalized trust region subproblem (GTRS) (Moré, 1993;Pong & Wolkowicz, 2014;Jiang & Li, 2019;Wang & Kılınc ¸-Karzan, 2020;Wang et al, 2021a), which minimizes a (possible nonconvex) quadratic function over a (possible nonconvex) quadratic inequality or equality constraint. Note that the TRS differs from the SCLS in the constraint, and the GTRS contains the SCLS as a special case.…”

“…for the SCLS (adapted from their analysis for the TRS). This indeed gives another Õ(N/ √ ǫ) time algorithm for solving the SPG-LS up to ǫ tolerance; see also Wang et al (2021a) for extensions of this idea to the GTRS.…”

“…We point out that the linear time algorithms for the TRS (Hazan & Koren, 2016;Ho-Nguyen & Kilinc-Karzan, 2017) can be adapted to design a linear time algorithm with complexity Õ(N/ √ ǫ) for the SCLS to achieve an ǫ optimal solution, and the linear time algorithms for the GTRS (Jiang & Li, 2020;Wang & Kılınc ¸-Karzan, 2020;Wang et al, 2021a) indicate that the SCLS, as a special case of the GTRS, can also be solved in linear time Õ(N/ √ ǫ). Here N denotes the number of nonzero entries in the data matrix, and the logarithm in the runtime comes from the probability of success in Lanczos type methods for finding the smallest eigenvalue of a matrix.…”