New Dynamic Orderings for the Parallel One–Sided Block-Jacobi SVD Algorithm

Abstract

Five variants of a new dynamic ordering are presented for the parallel one-sided block Jacobi SVD algorithm. Similarly to the two-sided algorithm, the dynamic ordering takes into account the actual status of a matrix-this time of its block columns with respect to their mutual orthogonality. Variants differ in the computational and communication complexities and in proposed global and local stopping criteria. Their performance is tested on a square random matrix of order 8192 with a random distribution of singular values using p = 16, 32, 64, 96 and 128 processors. All variants of dynamic ordering are compared with a parallel cyclic ordering, two-sided block-Jacobi method with dynamic ordering and the ScaLAPACK routine PDGESVD with respect to the number of parallel iteration steps needed for the convergence and total parallel execution time. Moreover, the relative errors in the orthogonality of computed left singular vectors and in the matrix assembled from computed singular triplets are also discussed. It turns out that the variant 3, for which a local optimality in some precisely defined sense can be proved, and its combination with variant 2, are the most efficient ones. For relatively small blocking factors = 2p, they outperform the ScaLAPACK procedure PDGESVD and are about 2 times faster.

Batched QR and SVD algorithms on GPUs with applications in hierarchical matrix compression

“…When we can no longer store the entire matrix in shared memory, we have to operate on the matrix in the slower global memory. Instead of repeatedly reading and updating the columns one at a time, block algorithms that facilitate cache reuse have been developed [20,21,22]. The main benefit of the block Jacobi algorithm is its high degree of parallelism; however, since we implement a batched routine for independent operations, we will use the serial block Jacobi algorithm for individual matrices and rely on the parallelism of the batch processing.…”

Performance analysis and optimization of the parallel one‐sided block Jacobi SVD algorithm with dynamic ordering and variable blocking

“…The algorithm consists of preprocessing (lines 1-3), iteration (lines [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19], and postprocessing (lines [20][21][22][23][24][25]. Here, > 0 is a predetermined convergence criterion and a r i denotes the ith column vector of A .r/ .…”

“…Table III, cited from [23], shows the number of iterations and sweeps of the OSBJ method with the exact and approximate weights. Table III, cited from [23], shows the number of iterations and sweeps of the OSBJ method with the exact and approximate weights.…”

Performance of the parallel block Jacobi method with dynamic ordering for the symmetric eigenvalue problem

“…Among the latter class of methods, the classical ordering, in which the off-diagonal block with the largest Frobenius norm is annihilated at each step, is expected to achieve faster convergence than other orderings, based on the analogy of the point Jacobi methods. This ordering has been extended by Bečka et al to parallel dynamic ordering [3,4], which chooses multiple off-diagonal blocks so that the sum of their squared Frobenius norms is maximal under the constraint that they can be annihilated simultaneously, and annihilates them in parallel. Although this ordering was originally proposed for the block Jacobi SVD (singular value decomposition) method, it should be promising also for the eigenvalue problem because it can attain both fast reduction of the off-diagonal norm and large-grain parallelism at the same time.…”