Fast Gradient Methods with Alignment for Symmetric Linear Systems without Using Cauchy Step

Abstract

The performance of gradient methods has been considerably improved by the introduction of delayed parameters. After two and a half decades, the revealing of secondorder information has recently given rise to the Cauchy-based methods with alignment, which reduce asymptotically the search spaces in smaller and smaller dimensions. They are generally considered as the state of the art of gradient methods. This paper reveals the spectral properties of minimal gradient and asymptotically optimal steps, and then suggests three fast methods with alignment without using the Cauchy step. The convergence results are provided, and numerical experiments show that the new methods provide competitive and more stable alternatives to the classical Cauchy-based methods. In particular, alignment gradient methods present advantages over the Krylov subspace methods in some situations, which makes them attractive in practice.

On Extensions of Limited Memory Steepest Descent Method

“…This might take advantage of both strategies and improvements might be expected. There exist other constant steplengths that could be exploited for the purpose of alignment, see [17] for more details.…”