New Fast and Accurate Jacobi SVD Algorithm. II

Drmač, Zlatko; Veselić, Krešimir

doi:10.1137/05063920x

Cited by 89 publications

(98 citation statements)

References 34 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On this latter goal, let us mention that algorithms for solving structured linear systems of equations more accurately than standard methods have been developed from the early days of numerical linear algebra [5] and many papers have been published on this topic since then (see the references in [17,29]). Although some preliminary ideas on accurate structured eigenvalue computations date from the 1960s [33], the systematic development of accurate algorithms for structured eigenvalue problems is much more recent, having started in the early 1990s with the celebrated paper [13], and has also received considerable attention (see [3,12,14,16,18,19,20,23,34,48,53] among other references). The present paper focuses on a part of accurate numerical linear algebra for which there are not many references available in the literature: algorithms for solving structured LS problems min x b − Ax 2 , where A ∈ C m×n and b ∈ C m , with much more accuracy than the one provided by standard algorithms and roughly with the same computational cost, that is, O(n 2 m) flops.…”

mentioning

confidence: 99%

Accurate Solution of Structured Least Squares Problems via Rank-Revealing Decompositions

Castro‐González¹,

Ceballos²,

Dopico³

et al. 2013

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

Abstract. Least squares problems minx b − Ax 2 where the matrix A ∈ C m×n (m ≥ n) has some particular structure arise frequently in applications. Polynomial data fitting is a well-known instance of problems that yield highly structured matrices, but many other examples exist. Very often, structured matrices have huge condition numbers κ 2 (A) = A 2 A † 2 (A † is the Moore-Penrose pseudoinverse of A) and therefore standard algorithms fail to compute accurate minimum 2-norm solutions of least squares problems. In this work, we introduce a framework that allows us to compute minimum 2-norm solutions of many classes of structured least squares problems accurately, i.e., with errors x 0 − x 0 2 / x 0 2 = O(u), where u is the unit roundoff, independently of the magnitude of κ 2 (A) for most vectors b. The cost of these accurate computations is O(n 2 m) flops, i.e., roughly the same cost as standard algorithms for least squares problems. The approach in this work relies in computing first an accurate rank-revealing decomposition of A, an idea that has been widely used in recent decades to compute, for structured ill-conditioned matrices, singular value decompositions, eigenvalues, and eigenvectors in the Hermitian case and solutions of linear systems with high relative accuracy. In order to prove that accurate solutions are computed, a new multiplicative perturbation theory of the least squares problem is needed. The results presented in this paper are valid for both full rank and rank deficient problems and also in the case of underdetermined linear systems (m < n). Among other types of matrices, the new method applies to rectangular Cauchy, Vandermonde, and graded matrices, and detailed numerical tests for Cauchy matrices are presented. 1. Introduction. Structured matrices arise frequently in theory and applications [44,45]. As a consequence, the design and analysis of special algorithms for performing structured matrix computations is a classical area of numerical linear algebra that attracts the attention of many researchers. Special algorithms for solving structured linear systems of equations or structured eigenvalue problems are included in many standard references [15,24,29,35,51], but special algorithms for solving structured least squares (LS) problems do not appear so often in the literature. The goal of special algorithms is to exploit the structure to increase the speed of computations, and/or to decrease storage requirements, and/or to improve the accuracy of the solutions with respect to standard algorithms. On this latter goal, let us mention that algorithms for solving structured linear systems of equations more accurately than standard methods have been developed from the early days of numerical linear

show abstract

mentioning

confidence: 99%

Accurate Solution of Structured Least Squares Problems via Rank-Revealing Decompositions

Castro‐González¹,

Ceballos²,

Dopico³

et al. 2013

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

show abstract

“…It may be possible to further improve those methods by restructuring the computation so that the wavefront algorithm for applying Givens rotations may be employed. -Periodically, the Jacobi iteration (for the EVD and SVD) receives renewed attention [Jacobi 1846;Demmel and Veselić 1992;Drmač 2009;Drmač and Veselić 2008a;2008b]. Since, like the QR algorithm, Jacobi-based approachs rely on Givens rotations, this work may further benefit those methods.…”

Section: Discussionmentioning

confidence: 99%

“…The authors assert that the "best method for computing all eigenvalues" is a derivative of the QR algorithm, known as the dqds algorithm [Fernando and Parlett 1994;Parlett and Marques 1999], but suggest that it is avoided only because it is, in practice, found to be "very slow." 4 Recent efforts at producing accurate SVD algorithms have focused on Jacobi iteration [Drmač and Veselić 2008a;2008b;Drmač 2009]. Admittedly, these algorithms tend to be somewhat more accurate than the QR algorithm [Demmel and Veselić 1992].…”

Section: Numerical Accuracymentioning

confidence: 99%

Restructuring the Tridiagonal and Bidiagonal QR Algorithms for Performance

Zee

Geijn

Quintana-Ortí

2014

ACM Trans. Math. Softw.

View full text Add to dashboard Cite

We show how both the tridiagonal and bidiagonal QR algorithms can be restructured so that they become rich in operations that can achieve near-peak performance on a modern processor. The key is a novel, cache-friendly algorithm for applying multiple sets of Givens rotations to the eigenvector/singular vector matrix. This algorithm is then implemented with optimizations that (1) leverage vector instruction units to increase floating-point throughput, and (2) fuse multiple rotations to decrease the total number of memory operations. We demonstrate the merits of these new QR algorithms for computing the Hermitian eigenvalue decomposition (EVD) and singular value decomposition (SVD) of dense matrices when all eigenvectors/singular vectors are computed. The approach yields vastly improved performance relative to the traditional QR algorithms for these problems and is competitive with two commonly used alternativesCuppen's Divide and Conquer algorithm and the Method of Multiple Relatively Robust Representationswhile inheriting the more modest O(n) workspace requirements of the original QR algorithms. Since the computations performed by the restructured algorithms remain essentially identical to those performed by the original methods, robust numerical properties are preserved.

show abstract

“…Some selected references in this area are [10][11][12][13][14][15][16][22][23][24]27,42]. By high relative accuracy of singular values we mean that the exact singular values σ i and their computed counterpartsσ i satisfy…”

Section: Introductionmentioning

confidence: 99%

Perturbation theory for the LDU factorization and accurate computations for diagonally dominant matrices

Dopico

Koev

2011

Numer. Math.

View full text Add to dashboard Cite

We present a structured perturbation theory for the LDU factorization of (row) diagonally dominant matrices and we use this theory to prove that a recent algorithm of Ye (Math Comp 77(264):2195-2230, 2008 computes the L , D and U factors of these matrices with relative errors less than 14n 3 u, where u is the unit roundoff and n × n is the size of the matrix. The relative errors for D are componentwise and for L and U are normwise with respect the "max norm" A M = max i j |a i j |. These error bounds guarantee that for any diagonally dominant matrix A we can compute accurately its singular value decomposition and the solution of the linear system Ax = b for most vectors b, independently of the magnitude of the traditional condition number of A and in O(n 3 ) flops. Mathematics Subject Classification (2000)

show abstract

New Fast and Accurate Jacobi SVD Algorithm. II

Cited by 89 publications

References 34 publications

Accurate Solution of Structured Least Squares Problems via Rank-Revealing Decompositions

Accurate Solution of Structured Least Squares Problems via Rank-Revealing Decompositions

Restructuring the Tridiagonal and Bidiagonal QR Algorithms for Performance

Perturbation theory for the LDU factorization and accurate computations for diagonally dominant matrices

Contact Info

Product

Resources

About