A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques

Bianconcini, Tommaso; Sciandrone, Marco

doi:10.1080/10556788.2016.1155213

Cited by 20 publications

(13 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We thus ignore them here, but note that this analysis also ensures global convergence to first-order stationary points. 8 Hence the subscript a, for "accurate".…”

Section: The Curtis-robinson-samadi Classmentioning

confidence: 99%

Worst-Case Evaluation Complexity and Optimality of Second-Order Methods for Nonconvex Smooth Optimization

Cartis

Gould

Toint

2019

Proceedings of the International Congress of Mathematicians (ICM 2018)

View full text Add to dashboard Cite

We establish or refute the optimality of inexact second-order methods for unconstrained nonconvex optimization from the point of view of worst-case evaluation complexity, improving and generalizing the results of [15,19]. To this aim, we consider a new general class of inexact second-order algorithms for unconstrained optimization that includes regularization and trust-region variations of Newton's method as well as of their linesearch variants. For each method in this class and arbitrary accuracy threshold ǫ ∈ (0, 1), we exhibit a smooth objective function with bounded range, whose gradient is globally Lipschitz continuous and whose Hessian is α−Hölder continuous (for given α ∈ [0, 1]), for which the method in question takes at least ⌊ǫ −(2+α)/(1+α) ⌋ function evaluations to generate a first iterate whose gradient is smaller than ǫ in norm. Moreover, we also construct another function on which Newton's takes ⌊ǫ −2 ⌋ evaluations, but whose Hessian is Lipschitz continuous on the path of iterates. These examples provide lower bounds on the worst-case evaluation complexity of methods in our class when applied to smooth problems satisfying the relevant assumptions. Furthermore, for α = 1, this lower bound is of the same order in ǫ as the upper bound on the worst-case evaluation complexity of the cubic regularization method and other methods in a class of methods proposed in [36] or in [65], thus implying that these methods have optimal worst-case evaluation complexity within a wider class of second-order methods, and that Newton's method is suboptimal.

show abstract

“…We thus ignore them here, but note that this analysis also ensures global convergence to first-order stationary points. 8 Hence the subscript a, for "accurate".…”

Section: The Curtis-robinson-samadi Classmentioning

confidence: 99%

Worst-Case Evaluation Complexity and Optimality of Second-Order Methods for Nonconvex Smooth Optimization

Cartis

Gould

Toint

2019

Proceedings of the International Congress of Mathematicians (ICM 2018)

View full text Add to dashboard Cite

show abstract

“…Proof. First, we observe that if λ = 0, then s = 0 is the only point that satisfies (2)- (3). So, in the following we consider the case in which λ > 0 (i.e., s = 0).…”

Section: Properties Of Stationary Pointsmentioning

confidence: 99%

“…where c ∈ R n , Q is a symmetric n × n matrix, σ is a positive real number and, here and in the rest of the article, · is the Euclidean norm. In recent years, there has been a growing interest in studying the properties of problem (1), since functions of the form of m(s) are used as local models (to be minimized) in many algorithmic frameworks for unconstrained optimization [14,18,19,17,6,7,12,1,2,4,11,3,5], which have been even extended to the constrained case [16,8,2]. To be more specific, let us consider the unconstrained optimization problem min x∈R n f (x), where f : R n → R is a twice continuously differentiable function.…”

Section: Introductionmentioning

confidence: 99%

On global minimizers of quadratic functions with cubic regularization

2018

View full text Add to dashboard Cite

In this paper, we analyze some theoretical properties of the problem of minimizing a quadratic function with a cubic regularization term, arising in many methods for unconstrained and constrained optimization that have been proposed in the last years. First we show that, given any stationary point that is not a global solution, it is possible to compute, in closed form, a new point with a smaller objective function value. Then, we prove that a global minimizer can be obtained by computing a finite number of stationary points. Finally, we extend these results to the case where stationary conditions are approximately satisfied, discussing some possible algorithmic applications.

show abstract

“…This research area has been remarkably active in recent years (see, for instance, [24,30,6,8,10,3,20,4,5,28,21,2,1,13]). Adaptive regularization algorithms, the class of methods considered here, compute steps from one iterate to the next by building and (often approximately) minimizing a model consisting of a truncated Taylor expansion of f , which is then "regularized" by adding a suitable power of the norm of the putative step.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Regularization Minimization Algorithms with Non-Smooth Norms and Euclidean Curvature

Gratton,

Toint

2021

Preprint

View full text Add to dashboard Cite

A regularization algorithm (AR1pGN) for unconstrained nonlinear minimization is considered, which uses a model consisting of a Taylor expansion of arbitrary degree and regularization term involving a possibly non-smooth norm. It is shown that the nonsmoothness of the norm does not affect the O( −(p+1)/p 1 ) upper bound on evaluation complexity for finding first-order 1 -approximate minimizers using p derivatives, and that this result does not hinge on the equivalence of norms in IR n . It is also shown that, if p = 2, the bound of O( −32 ) evaluations for finding second-order 2 -approximate minimizers still holds for a variant of AR1pGN named AR2GN, despite the possibly non-smooth nature of the regularization term. Moreover, the adaptation of the existing theory for handling the non-smoothness results in an interesting modification of the subproblem termination rules, leading to an even more compact complexity analysis. In particular, it is shown when the Newton's step is acceptable for an adaptive regularization method. The approximate minimization of quadratic polynomials regularized with non-smooth norms is then discussed, and a new approximate second-order necessary optimality condition is derived for this case. An specialized algorithm is then proposed to enforce the first-and second-order conditions that are strong enough to ensure the existence of a suitable step in AR1pGN (when p = 2) and in AR2GN, and its iteration complexity is analyzed.

show abstract

A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques

Cited by 20 publications

References 21 publications

Worst-Case Evaluation Complexity and Optimality of Second-Order Methods for Nonconvex Smooth Optimization

Worst-Case Evaluation Complexity and Optimality of Second-Order Methods for Nonconvex Smooth Optimization

On global minimizers of quadratic functions with cubic regularization

Adaptive Regularization Minimization Algorithms with Non-Smooth Norms and Euclidean Curvature

Contact Info

Product

Resources

About