Descent Spectral Versions of the Traditional Conjugate Gradient Algorithms With Application to Nonnegative Matrix Factorization

Abstract

Despite computational superiorities, some traditional conjugate gradient algorithms such as Polak–Ribiére–Polyak and Hestenes–Stiefel methods generally fail to guarantee the descent condition. Here, in a matrix viewpoint, spectral versions of such methods are developed which fulfill the descent condition. The convergence of the given spectral algorithms is argued briefly. Afterwards, we propose an improved version of the nonnegative matrix factorization problem by adding penalty terms to the model, for controlling the condition number of one of the factorization elements. Finally, the computational merits of the method are examined using a set of CUTEr test problems as well as some random nonnegative matrix factorization models. The results typically agree with our analytical spectrum.