A Linear-Time Algorithm for Generalized Trust Region Subproblems

Abstract

In this paper, we provide the first provable linear-time (in term of the number of non-zero entries of the input) algorithm for approximately solving the generalized trust region subproblem (GTRS) of minimizing a quadratic function over a quadratic constraint under some regularity condition. Our algorithm is motivated by and extends a recent linear-time algorithm for the trust region subproblem by Hazan and Koren [Math. Program., 2016, 158(1-2): 363-381]. However, due to the non-convexity and non-compactness of the feasible region, such an extension is nontrivial. Our main contribution is to demonstrate that under some regularity condition, the optimal solution is in a compact and convex set and lower and upper bounds of the optimal value can be computed in linear time. Using these properties, we develop a linear-time algorithm for the GTRS.

The generalized trust region subproblem: solution complexity and convex hull results

“…Comparing ( 6) and (2), we see that our running times match the dependences on N , n, , and p from the algorithm for the TRS presented by Hazan and Koren [13]. Comparing (6) (specifically the running time for finding an -optimal solution) and (3), we see that our running time matches the linear dependence on N and improves the dependence on by a logarithmic factor from the running time presented by Jiang and Li [18]. The dependences on the regularity parameters in the two running times are incomparable (see Remark 8) but there exist examples where our running time gives a polynomialorder improvement upon the running time presented by Jiang and Li [18] (see Remark 12).…”

“…In comparison to the approach taken by Jiang and Li [18], we believe our approach is conceptually simpler and more straightforward to implement. In particular our approach directly solves the GTRS in the primal space as opposed to solving a feasibility version of the dual problem.…”

“…The TRS is an essential ingredient of trust-region methods that are commonly used to solve continuous nonconvex optimization problems [6,28,30] and also arises in applications such as robust optimization [2,14]. On the other hand, the GTRS has applications in nonconvex quadratic integer programs, signal processing, and compressed sensing; see [1,4,18] and references therein for more applications.…”

“…In particular, theoretical guarantees have not yet been given regarding how these algorithms perform when only approximate generalized eigenvalue computations are available. This is of interest as, in practice, we cannot hope to numerically compute generalized eigenvalues exactly; see also the discussion in Section 4.1 in [18]. We would like to remark that numerical experiments in these papers [1,17,30] have suggested that algorithms motivated by these ideas may perform well even using only approximate generalized eigenvalue computations.…”

“…The very recent work of Jiang and Li [18] presents an algorithm for solving the GTRS up to an additive error in the objective with high probability under the regularity condition. This algorithm relies on machinery developed by [13] for solving the TRS and differs from previous algorithms in that it does not assume the ability to compute a simultaneously-diagonalizing basis or generalized eigenvalues.…”