Random multipliers numerically stabilize Gaussian and block Gaussian elimination: Proofs and an extension to low-rank approximation

Pan, Victor Y.; Qian, Guoliang; Yan, Xiaodong

doi:10.1016/j.laa.2015.04.021

Cited by 19 publications

(32 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This was also the case in the previous studies of GENP in [PQZ13], [BDHT13], [PQY15], and [DDF14], but our present tests cover pre-processing also with multipliers from new classes as well as by means of augmentation.…”

Section: Contribution: Numerical Testssupporting

confidence: 51%

“…Having specific bad pairs of inputs and multipliers does not contradict claim (ii) of Corollary 4.1, and actually in extensive tests in [PQZ13] and [PQY15] very good numerical stability has been observed when we applied GENP to various classes of nonsingular well-conditioned input matrices with random circulant multipliers.…”

Section: Proof Hereafter Det(αmentioning

confidence: 63%

“…This property is important for n × n input matrices of smaller sizes: refinement iteration involves O(n 2 ) flops versus cubic cost of 2 3 n 3 flops, involved in Gaussian elimination, and for small n quadratic cost of refinement can make up a large share of the overall cost. A single refinement iteration was always sufficient (and more frequently was not even needed) in our tests in order to match or to exceed the output accuracy of GEPP (see also similar empirical data in [PQZ13], [BDHT13], [DDF14], and [PQY15] and see [H02, Chapter 12], [GL13, Section 3.5.3], and the references therein for detailed coverage of iterative refinement).…”

Section: By Virtue Of Lemma 21) and Vice Versa Each Of The Two Vectmentioning

confidence: 99%

“…Gaussian matrices have been generated by applying the standard normal distribution function randn of MATLAB. We refer the reader to [PQZ13], [PQY15], and [PZa] for other extensive tests of GENP with randomized pre-processing.…”

Section: Numerical Experimentsmentioning

confidence: 99%

See 3 more Smart Citations

Numerically safe Gaussian elimination with no pivoting

Pan¹,

Zhao²

2017

Linear Algebra and its Applications

Self Cite

View full text Add to dashboard Cite

Gaussian elimination with no pivoting and block Gaussian elimination are attractive alternatives to the customary but communication intensive Gaussian elimination with partial pivoting 1 provided that the computations proceed safely and numerically safely, that is, run into neither division by 0 nor numerical problems. Empirically, safety and numerical safety of GENP have been consistently observed in a number of papers where an input matrix was pre-processed with various structured multipliers chosen ad hoc. Our present paper provides missing formal support for this empirical observation and explains why it was elusive so far. Namely we prove that GENP is numerically unsafe for a specific class of input matrices in spite of its pre-processing with some well-known and well-tested structured multipliers, but we also prove that GENP and BGE are safe and numerically safe for the average input matrix pre-processed with any nonsingular and well-conditioned multiplier. This should embolden search for sparse and structured multipliers, and we list and test some new classes of them. We also seek randomized pre-processing that universally (that is, for all input matrices) supports (i) safe GENP and BGE with probability 1 and/or (ii) numerically safe GENP and BGE with a probability close to 1. We achieve goal (i) with a Gaussian structured multiplier and goal (ii) with a Gaussian unstructured multiplier and alternatively with Gaussian structured augmentation. We consistently confirm all these formal results with our tests of GENP for benchmark inputs. We have extended our approach to other fundamental matrix computations and keep working on further extensions.2000 Math. Subject Classification: 15A06, 15A52, 15A12, 65F05, 65F35

show abstract

Section: Contribution: Numerical Testssupporting

confidence: 51%

Section: Proof Hereafter Det(αmentioning

confidence: 63%

Section: By Virtue Of Lemma 21) and Vice Versa Each Of The Two Vectmentioning

confidence: 99%

Section: Numerical Experimentsmentioning

confidence: 99%

See 2 more Smart Citations

Numerically safe Gaussian elimination with no pivoting

Pan¹,

Zhao²

2017

Linear Algebra and its Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…[HMT11]) and with randomized preprocessing of Gaussian elimination without pivoting 1 in [PQY15]. In particular, similarly to randomized low-rank approximation in [HMT11, Section 11], our techniques, algorithms and their analysis can be extended to the case where preprocessing with Gaussian matrices is replaced by preprocessing with Semisample Random Fourier Transform 2 structured matrices, defined in our Appendix C and [HMT11, Section 11].…”

Section: Randomized Sparse and Structured Preprocessingmentioning

confidence: 99%

New studies of randomized augmentation and additive preprocessing

Pan

Zhao

2017

Linear Algebra and its Applications

Self Cite

View full text Add to dashboard Cite

Abstract• A standard Gaussian random matrix has full rank with probability 1 and is wellconditioned with a probability quite close to 1 and converging to 1 fast as the matrix deviates from square shape and becomes more rectangular.• If we append sufficiently many standard Gaussian random rows or columns to any matrix A, such that ||A|| = 1, then the augmented matrix has full rank with probability 1 and is well-conditioned with a probability close to 1, even if the matrix A is rank deficient or ill-conditioned.• We specify and prove these properties of augmentation and extend them to additive preprocessing, that is, to adding a product of two rectangular Gaussian matrices.• By applying our randomization techniques to a matrix that has numerical rank ρ, we accelerate the known algorithms for the approximation of its leading and trailing singular spaces associated with its ρ largest and with all its remaining singular values, respectively.• Our algorithms use much fewer random parameters and run much faster when various random sparse and structured preprocessors replace Gaussian. Empirically the outputs of the resulting algorithms is as accurate as the outputs under Gaussian preprocessing.• Our novel duality techniques provides formal support, so far missing, for these empirical observations and opens door to derandomization of our preprocessing and to further acceleration and simplification of our algorithms by using more efficient sparse and structured preprocessors.• Our techniques and our progress can be applied to various other fundamental matrix computations such as the celebrated low-rank approximation of a matrix by means of random sampling. 1Key Words: Randomized matrix algorithms; Gaussian random matrices; Singular spaces of a matrix; Duality; Derandomization; Sparse and structured preprocessors 1 Introduction Randomized augmentation: outlineA standard Gaussian m × n random matrix, G (hereafter referred to just as Gaussian), has full rank with probability 1 (see Theorem B.1). Furthermore the expected spectral norms ||G|| and ||G + ||, G + denoting the Moore-Penrose generalized inverse, satisfy the following estimates (see Theorems B.2 and B.3):• E(||G||) ≈ 2 √ h, for h = max{m, n}, and|m−n| provided that l = min{m, n}, m = n, and e = 2.71828 . . . . Thus, for moderate or reasonably large integers m and n, the matrix G can be considered wellconditioned with the confidence growing fast as the integer |m − n| increases from 0. By virtue of part 2 of Theorem B.3, the matrix G can be viewed as well-conditioned even for m = n, although with a grain of salt, depending on context. Motivated by this information, we append sufficiently but reasonably many Gaussian rows or columns to any matrix A, possibly rank deficient or ill-conditioned, but normalized, such that ||A|| = 1. (Our approach requires attention to various pitfalls, and in particular it fails without normalization of an input matrix.) Then we prove that the cited properties of a Gaussian matrix also hold for the augmented matrix K and similarly for th...

show abstract

CUR LRA at Sublinear Cost Based on Volume Maximization

Luan

Pan

2020

Mathematical Aspects of Computer and Information Sciences

View full text Add to dashboard Cite

Random multipliers numerically stabilize Gaussian and block Gaussian elimination: Proofs and an extension to low-rank approximation

Cited by 19 publications

References 46 publications

Numerically safe Gaussian elimination with no pivoting

Numerically safe Gaussian elimination with no pivoting

New studies of randomized augmentation and additive preprocessing

CUR LRA at Sublinear Cost Based on Volume Maximization

Contact Info

Product

Resources

About