On the use of iterative methods in cubic regularization for unconstrained optimization

Bianconcini, Tommaso; Liuzzi, Giampaolo; Morini, Benedetta; Sciandrone, Marco

doi:10.1007/s10589-014-9672-x

Cited by 41 publications

(27 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where σ t is the cubic regularization parameter chosen for the current iteration. As in the case of TR, the major bottleneck of CR involved solving the sub-problem (2b), for which various techniques have been proposed, e.g., [1,4,8,9]. To the best of our knowledge, the use of such regularization, was first introduced in the pioneering work of [34], and subsequently further studied in the seminal works of [9,10,45].From the worst-case complexity point of view, CR has a better dependence on ǫ g compared to TR.…”

Section: Cubic Regularizationmentioning

confidence: 99%

Newton-type methods for non-convex optimization under inexact Hessian information

2019

View full text Add to dashboard Cite

We consider variants of trust-region and adaptive cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under certain condition on the inexact Hessian, and using approximate solution of the corresponding sub-problems, we provide iteration complexity to achieve ǫ-approximate second-order optimality which have been shown to be tight. Our Hessian approximation condition offers a range of advantages as compared with the prior works and allows for direct construction of the approximate Hessian with a priori guarantees through various techniques, including randomized sampling methods. In this light, we consider the canonical problem of finite-sum minimization, provide appropriate uniform and non-uniform sub-sampling strategies to construct such Hessian approximations, and obtain optimal iteration complexity for the corresponding subsampled trust-region and adaptive cubic regularization methods.

show abstract

Section: Cubic Regularizationmentioning

confidence: 99%

Newton-type methods for non-convex optimization under inexact Hessian information

2019

View full text Add to dashboard Cite

show abstract

“…A.1 for the proof conditional on these lemmas). For the first lemma, let κ ∈ R d satisfy κ (1) ≤ κ (2) ≤ . .…”

Section: Appendixmentioning

confidence: 99%

Gradient Descent Finds the Cubic-Regularized Nonconvex Newton Step

Carmon¹,

Duchi²

2019

SIAM J. Optim.

View full text Add to dashboard Cite

We consider the minimization of non-convex quadratic forms regularized by a cubic term, which exhibit multiple saddle points and poor local minima. Nonetheless, we prove that, under mild assumptions, gradient descent approximates the global minimum to within ε accuracy in O(ε −1 log(1/ε)) steps for large ε and O(log(1/ε)) steps for small ε (compared to a condition number we define), with at most logarithmic dependence on the problem dimension. When we use gradient descent to approximate the Nesterov-Polyak cubic-regularized Newton step, our result implies a rate of convergence to second-order stationary points of general smooth non-convex functions.

show abstract

“…where σ > 0 p > 2 and · is the Euclidean norm-note that Q is bounded below over IR n , and the global minimizer is well defined. Such methods have been advocated by a number of authors, e.g., [1][2][3]7]. Here we are interested in how the norms of the estimates of the solution, and the corresponding "multipliers" σ x p−2 , evolve as the Krylov process proceeds.…”

Section: Introductionmentioning

confidence: 99%

On monotonic estimates of the norm of the minimizers of regularized quadratic functions in Krylov spaces

2019

View full text Add to dashboard Cite

show abstract

On the use of iterative methods in cubic regularization for unconstrained optimization

Cited by 41 publications

References 17 publications

Newton-type methods for non-convex optimization under inexact Hessian information

Newton-type methods for non-convex optimization under inexact Hessian information

Gradient Descent Finds the Cubic-Regularized Nonconvex Newton Step

On monotonic estimates of the norm of the minimizers of regularized quadratic functions in Krylov spaces

Contact Info

Product

Resources

About