Reducing Inner Product Computation in the Parallel One-Sided Jacobi Algorithm

Abstract

IntroductionOne drawback of the cyclic one-sided Jacobi algorithm is the necessity of computing the inner product of each column pair to be rotated-indeed, just to check whether the pair needs to be rotated. However, if the orthogonality of a column pair known to be orthogonal during a sweep is not subsequently destroyed before being encountered during the next sweep, then the inner product computation in the latter sweep is unnecessary. The number of such inner products becomes significant as the process nears convergence. To avoid these unnecessary inner products, the usual algorithm is extended to include a data structure which keeps a record of the current orthogonality status of the column pairs.In the parallel setting, the reduction during the latter sweeps in both the number of column pairs rotated and the number of inner product computations in a sweep will not generally be uniformly distributed across the processors. Because of the resulting loss of load balance, the actual improvement in runtimes will be less than one would otherwise expect. A statistical analysis of this phenomenon is given.The performance of the enhanced algorithm is observed through implementation on the 128-node Intel iPSC/860 at the Oak Ridge National Laboratory. A discussion of the correlation of these results with the statistical analysis mentioned above is also given.It was observed some time ago by Sameh [12] and Luk [7] that the one-sided Jacobi algorithm was wellsuited for singular value and eigenvalue computation in a multiprocessor environment. More recently it has been observed [1,4,5,6,10] that the one-sided Jacobi algorithm is especially naturally suited for such computations on distributed memory and vector architectures. In the distributed memory environment, its natural parallelism permits excellent load balancing and the required message passing is relatively small. Because of its richness in vector operations, it is naturally suited for vector architectures.This has led to a renewed interest in the one-sided Jacobi algorithm. Berry and Sameh [l] have effectively implemented it in a multiprocessor environment. Eberlein [4,5] and Eberlein and Park [6,8] have done so on distributed memory machines. Rat11 [9] and de Rijk [lo] have given a fast Givens-type variant of the one-sided Jacobi algorithm which reduces the operation count and permits more effective vectorization.The Jacobi algorithms have other advantages. It has recently been shown by Demmel and VeseliC [3] that the Jacobi algorithms compute small eigenvalues and singular values and small components of eigenvectors and singular vectors with higher relative accuracy than either the QR and Golub-Reinsch algorithms or the divide and conquer method.One drawback of the cyclic one-sided Jacobi algorithm is the necessity of computing the inner product of each column pair to be rotated-indeed, just to check whether the pair needs to be rotated. However, if the orthogonality of a column pair known to be orthogonal during a sweep is not subsequently destroyed before b...