Conjugate direction methods in optimization

Hestenes, Magnus R.

doi:10.1007/bfb0007220

Cited by 45 publications

(82 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This special property of C enables us to incorporate a sparsity preserving quadratic programming algorithm, such as the SOR algorithm [10] and the conjugate gradient algorithm [7,9], in the algorithm SLM presented in Section 2.…”

Section: Sparsity Preserving Algorithms For Solving Subproblemsmentioning

confidence: 99%

“…One is a slight modi¡ of the conjugate gradient (CG) algorithm [7] and the other is the successive over-relaxation (SOR) algorithm [10] for solving convex quadratic programming problems with simple bounds.…”

Section: Sparsity Preserving Algorithms For Solving Subproblemsmentioning

confidence: 99%

“…In fact, if those methods are used to solve the subproblems, the sparsity of the original problem is preserved because they require only simple operations on the rows of the constraint matrix. Thus, large-scale problems can be dealt with when the constraint matrices are sparse enough to be stored in a compact form [7,10].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Successive linearization methods for large-scale nonlinear programming problems

Fukushima

Takazawa²,

Ohsaki

et al. 1992

Japan J. Indust. Appl. Math.

View full text Add to dashboard Cite

We propose a spavsity preserving algorithm for solving large-scale, nonlineav programming problems. The algorithm solves at each iteration a subproblem, which contalns a lineavized objective function augmented by a simple quadratic term and linearized constraints. The quadratic term added to the lineavized objective function plays the role of step restriction which is essential in ensuring global convergence of the algorithm. Ir the conjugate gradient method or successive over-relaxation method is used to solve the subproblems, the spavsity of the original problem is preserved, because those methods only require simple operations on the rows of the constraint matrix. Thus, lavge-scale problems can be dealt with when the constraint matrices ave sparse enough to be stored in a compact form. Praztical implementation of the algorithm is described and computational results ave reported.Large-scale nonlinear programming problems usually have the property that many of their problem functions are linear and sparse. Thus, successive linear programming (SLP) algorithms [4,12] have been preferred for solving them fora long time, in spite of the lack of theoretical convergence properties. In a recent paper [17], Zhang, Kim and Lasdon propose a globally convergent SLP algorithm in which an exact penalty function technique is incorporated with a trust region method. Though SLP algorithms can solve very large problems [1], they may not completely preserve the sparsity which the original problems may have, because of possible fill-in caused in solving LP subproblems.The purpose of this paper is to propose a sparsity preserving algorithm, which is also globally convergent to an optimal solution of the problem. The proposed algorithm solves at each iteration a subproblem which contains a linearized objective function augmented by a simple quadratic term and linearized constraints. Though this algorithm resembles successive quadratic programming (SQP) algorithms, it is based on an idea entirely different from typical SQP methods such as [3,5,13,14]. In our algorithm, the quadratic term added to the linearized objective

show abstract

Section: Sparsity Preserving Algorithms For Solving Subproblemsmentioning

confidence: 99%

Section: Sparsity Preserving Algorithms For Solving Subproblemsmentioning

confidence: 99%

See 1 more Smart Citation

Successive linearization methods for large-scale nonlinear programming problems

Fukushima

Takazawa²,

Ohsaki

et al. 1992

Japan J. Indust. Appl. Math.

View full text Add to dashboard Cite

show abstract

“…If the function is quadratic, and if the line search is exact, the vector d k+1 can be proved to be also the shortest residual in the k-simplex whose vertices are −g 1 , · · ·, −g k+1 (see [2]). …”

Section: Introductionmentioning

confidence: 99%

“…The SR method was presented by Hestenes in his monograph [2] on conjugate direction methods. In the SR method the search direction d k is taken as the shortest vector of the form (1.6) where d 1 = −g 1 .…”

Section: Introductionmentioning

confidence: 99%

Global convergence of the method of shortest residuals

Dai

Yuan

1999

Numer. Math.

View full text Add to dashboard Cite

The method of shortest residuals (SR) was presented by Hestenes and studied by Pytlak. If the function is quadratic, and if the line search is exact, then the SR method reduces to the linear conjugate gradient method. In this paper, we put forward the formulation of the SR method when the line search is inexact. We prove that, if stepsizes satisfy the strong Wolfe conditions, both the Fletcher-Reeves and Polak-Ribière-Polyak versions of the SR method converge globally. When the Wolfe conditions are used, the two versions are also convergent provided that the stepsizes are uniformly bounded; if the stepsizes are not bounded, an example is constructed to show that they need not converge. Numerical results show that the SR method is a promising alternative of the standard nonlinear conjugate gradient method. Subject Classifications (1991): 65k, 90c Mathematics

show abstract