In unconstrained minimization of a function/, the method of Davidon-Fleteher-Powell (a "variable-metric" method) enables the inverse of the Hessian H of / to be approximated step wise, using only values of the gradient of/. It is shown here that, by solving a certain variational problem, formulas for the successive corrections to H can be derived which closely resemble Davidon's. A symmetric correction matrix is sought which minimizes a weighted Euclidean norm, and also satisfies the "DFP condition." Numerical tests are described, comparing the performance (on four "standard" test functions) of two variationally-derived formulas with Davidon's. A proof by Y. Bard, modelled on Fletcher and Powell's, showing that the new formulas give the exact H after N steps, is included in an appendix.

No abstract

A comparison is made among various gradient methods for maximizing a function, based on a characterization by Crockett and Chernoff of the class of these methods. By defining the “efficiency” of a gradient step in a certain way, it becomes easy to compare the efficiencies of different schemes with that of Newton’s method, which can be regarded as a particular gradient scheme. For quadratic functions, it is shown that Newton’s method is the most efficient (a conclusion which may be approximately true for nonquadratic functions). For functions which are not concave (downward), it is shown that the Newton direction may be just the opposite of the most desirable one. A simple way of correcting this is explained.

No abstract

No abstract