Preconditioning Newton–Krylov methods in nonconvex large scale optimization

Fasano, Giovanni; Roma, Massimo

doi:10.1007/s10589-013-9563-6

Cited by 25 publications

(45 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [12], besides some theoretical properties, the results of an extensive numerical experience is reported, showing that the use of such preconditioners makes PNCG algorithms more efficient and robust than the unpreconditioned ones, on most CUTEst [18] large scale problems. These preconditioners are also inspired by some recent proposals in the context of Newton-Krylov methods (see [14] [15]), along with some effective preconditioning techniques from the literature of preconditioners for symmetric linear systems, namely the Limited Memory Preconditioners [19].…”

Section: The Class Of Preconditionersmentioning

confidence: 99%

Exploiting damped techniques for nonlinear conjugate gradient methods

et al. 2017

Self Cite

View full text Add to dashboard Cite

In this paper we propose the use of damped techniques within Nonlinear Conjugate Gradient (NCG) methods. Damped techniques were introduced by Powell and recently reproposed by Al-Baali and till now, only applied in the framework of quasi-Newton methods. We extend their use to NCG methods in large scale unconstrained optimization, aiming at possibly improving the efficiency and the robustness of the latter methods, especially when solving difficult problems. We consider both unpreconditioned and Preconditioned NCG (PNCG). In the latter case, we embed damped techniques within a class of preconditioners based on quasi-Newton updates. Our purpose is to possibly provide efficient preconditioners which approximate, in some sense, the inverse of the Hessian matrix, while still preserving information provided by the secant equation or some of its modifications. The results of an extensive numerical experience highlights that the proposed approach is quite promising.

show abstract

Section: The Class Of Preconditionersmentioning

confidence: 99%

Exploiting damped techniques for nonlinear conjugate gradient methods

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, we highlight that item (i) in the previous proposition does not imply also that the point (x k , x 0k ) T , with y k = x k /x 0k , is in the asymptotic cone of F γ . Indeed (i) in general implies the situation depicted in Figure 11 (left), where (p k , 0) T ∈ C ∞ , but the point y k does not satisfy equation (18). Figure 11 (right) gives a graphical representation of item (ii) of Proposition 8.3.…”

Section: Cg Iterations and Cg Failure: A Geometric Viewpointmentioning

confidence: 97%

“…Indeed, the results of Lemma 7.1 suggest that in Cartesian coordinates the polar hyperplane of a point not only includes first order information (i.e. the gradient of the function at the current iterate), but also information on the 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 Of course, since possibly y k in Table 1 does not satisfy equation (18), in general in Cartesian coordinates the line y k + λp k , λ ∈ R, does not include the centre y * of F. Unfortunately, the CG is unable to compute a finite steplength along p k (see also Figure 11), so that it stops prematurely. Nevertheless, an ad hoc inexact linesearch procedure along p k could be conceived.…”

Section: Perspectivesmentioning

confidence: 99%

See 1 more Smart Citation

Conjugate Direction Methods and Polarity for Quadratic Hypersurfaces

Fasano

Pesenti

2017

J Optim Theory Appl

Self Cite

View full text Add to dashboard Cite

Abstract:We use some results from polarity theory to recast several geometric properties of Conjugate Gradient-based methods, for the solution of nonsingular symmetric linear systems. This approach allows us to pursue three main theoretical objectives. First, we can provide a novel geometric perspective on the generation of conjugate directions, in the context of positive definite systems. Second, we can extend the above geometric perspective to treat the generation of conjugate directions for handling indefinite linear systems. Third, by exploiting the geometric insight suggested by polarity theory, we can easily study the possible degeneracy (pivot breakdown) of Conjugate Gradientbased methods on indefinite linear systems. In particular, we prove that the degeneracy of the standard Conjugate Gradient on nonsingular indefinite linear systems can occur only once in the execution of the Conjugate Gradient. Once again we strongly appreciated the suggestions by the Editorial Board and the anonymous Referees, which definitely contributed to improve and enhance the paper. Hereafter we reply to the observations raised by each of them. We also remark that the paper has been given to a professional proofreader before resubmission: we hope we were able to comply with all the issues raised by the Reviewers. Powered by Editorial Manager® and ProduXion Manager® from Aries Systems CorporationAnswer to the Editor in Chief and the Associate EditorAuthors: The instructions of the Editor/Associate Editor should have been followed, including both structural modifications (e.g. we splitted the former section of Conclusions) and minor suggestions.Answer to the First Referee Authors: All this Reviewer's comments should have been included, following her/his indications. In addition, an English native speaker has contributed with proofreading.Answer to the Second Referee Authors: All the accurate Reviewer's modifications should have been implemented, following her/his indications. Furthermore, we also did our best to revise the entire paper, on the basis of the specific comments of this Reviewer. This has led to some additional small changes, including punctuation, with respect to the previous version of the paper. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 2 Giovanni Fasano, Raffaele Pesenti by exploiting the geometric insight suggested by polarity theory, we can easily study the possible degeneracy (pivot breakdown) of Conjugate Gradient-based methods on indefinite linear systems. In particular, we prove that the degeneracy of the standard Conjugate Gradient on nonsingular indefinite linear systems can occur only once in the execution of the Conjugate Gradient. Powered by Editorial Manager® and ProduXion Manager® from Aries Systems Corporation

show abstract

“…Similarly, matrix-free preconditioning for linear systems or sequences of linear systems is currently an appealing research topic, too (see e.g. [4,5,13,14] ).…”

Section: Introductionmentioning

confidence: 99%

Preconditioned Nonlinear Conjugate Gradient methods based on a modified secant equation

Caliciotti

Fasano

Roma

2018

Applied Mathematics and Computation

Self Cite

View full text Add to dashboard Cite

a b s t r a c tThis paper includes a twofold result for the Nonlinear Conjugate Gradient (NCG) method, in large scale unconstrained optimization. First we consider a theoretical analysis, where preconditioning is embedded in a strong convergence framework of an NCG method from the literature. Mild conditions to be satisfied by the preconditioners are defined, in order to preserve NCG convergence. As a second task, we also detail the use of novel matrix-free preconditioners for NCG. Our proposals are based on quasi-Newton updates, and either satisfy the secant equation or a secant-like condition at some of the previous iterates. We show that, in some sense, the preconditioners we propose also approximate the inverse of the Hessian matrix. In particular, the structures of our preconditioners depend on lowrank updates used, along with different choices of specific parameters. The low-rank updates are obtained as by-product of NCG iterations. The results of an extended numerical experience using large scale CUTEst problems is reported, showing that our preconditioners can considerably improve the performance of NCG methods.

show abstract

Preconditioning Newton–Krylov methods in nonconvex large scale optimization

Cited by 25 publications

References 28 publications

Exploiting damped techniques for nonlinear conjugate gradient methods

Exploiting damped techniques for nonlinear conjugate gradient methods

Conjugate Direction Methods and Polarity for Quadratic Hypersurfaces

Preconditioned Nonlinear Conjugate Gradient methods based on a modified secant equation

Contact Info

Product

Resources

About