An Implicit Representation and Iterative Solution of Randomly Sketched Linear Systems

Patel, Vivak; Jahangoshahi, Mohammad; Maldonado, Daniel Adrian

doi:10.1137/19m1259481

Cited by 6 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A different simple yet surprisingly effective modification of the probability distribution at each iteration is sampling without replacement 21,22 . In these methods a row can only be sampled again after all of the other rows of the matrix have been sampled.…”

Section: Discussionmentioning

confidence: 99%

“…A different simple yet surprisingly effective modification of the probability distribution at each iteration is sampling without replacement. 21,22 In these methods a row can only be sampled again after all of the other rows of the matrix have been sampled. This contrasts the standard method of sampling with replacement in which there is a fixed probability for a row to be selected at each iteration throughout the duration of the algorithm.…”

Section: Future Directionsmentioning

confidence: 99%

See 1 more Smart Citation

Selectable Set Randomized Kaczmarz

Yaniv

Moorman

Swartworth

et al. 2022

Numerical Linear Algebra App

View full text Add to dashboard Cite

The Randomized Kaczmarz method (RK) is a stochastic iterative method for solving linear systems that has recently grown in popularity due to its speed and low memory requirement. Selectable Set Randomized Kaczmarz is a variant of RK that leverages existing information about the Kaczmarz iterate to identify an adaptive "selectable set" and thus yields an improved convergence guarantee. In this article, we propose a general perspective for selectable set approaches and prove a convergence result for that framework. In addition, we define two specific selectable set sampling strategies that have competitive convergence guarantees to those of other variants of RK. One selectable set sampling strategy leverages information about the previous iterate, while the other leverages the orthogonality structure of the problem via the Gramian matrix. We complement our theoretical results with numerical experiments that compare our proposed rules with those existing in the literature.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Future Directionsmentioning

confidence: 99%

Selectable Set Randomized Kaczmarz

Yaniv

Moorman

Swartworth

et al. 2022

Numerical Linear Algebra App

View full text Add to dashboard Cite

show abstract

mentioning

confidence: 94%

“…In this work, we address the shortcomings of these previous works [12,27,26]. Specifically, we refine the three properties that we presented in [27,26] to be more precise for the vector case and to generalize to the block case, which allows our theory to cover the specific methods presented in [12] and many more (see section 4). Thus, our theory enables the rigorous development and deployment of novel randomized solvers that take advantage of efficient block operations and that can be tailored to specific problems and computing environments.…”

mentioning

confidence: 94%

“…In furtherance of this effort, we previously developed a novel framework for designing and analyzing the important class of randomized vector adaptive linear system solvers [27,26], which unified the analysis of such vector linear solvers as independent and identically distributed sampling randomized solvers [35,39,13,34]; greedy deterministic solvers [2,20]; randomized and deterministic cyclic sampling solvers [27]; greedy subsets followed by random selection solvers [5]; randomized subsets followed by greedy selection solvers [14]; and streaming solvers. More importantly, our framework and corresponding theory has displaced the need for the individual analysis of randomized vector adaptive solver, which enables the rapid prototyping and deployment of mathematically-sound randomized vector adaptive solvers that are highly-tailored to each unique problem and hardware environment.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Randomized Block Adaptive Linear System Solvers

Patel¹,

Jahangoshahi²,

Maldonado³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Randomized linear solvers leverage randomization to structure-blindly compress and solve a linear system to produce an inexpensive solution. While such a property is highly desirable, randomized linear solvers often suffer when it comes to performance as either (1) problem structure is not being exploited, and (2) hardware is inefficiently used. Thus, randomized adaptive solvers are starting to appear that use the benefits of randomness while attempting to still exploit problem structure and reduce hardware inefficiencies. Unfortunately, such randomized adaptive solvers are likely to be without a theoretical foundation to show that they will work (i.e., find a solution). Accordingly, here, we distill three general criteria for randomized block adaptive solvers, which, as we show, will guarantee convergence of the randomized adaptive solver and supply a worst-case rate of convergence. We will demonstrate that these results apply to existing randomized block adaptive solvers, and to several that we devise for demonstrative purposes.

show abstract

A weighted randomized sparse Kaczmarz method for solving linear systems

et al. 2022

View full text Add to dashboard Cite

The randomized sparse Kaczmarz method, designed for seeking the sparse solutions of the linear systems Ax = b, selects the i-th projection hyperplane with likelihood proportional to ∥ai∥ 2 2 , where a T i is i-th row of A. In this work, we propose a weighted randomized sparse Kaczmarz method, which selects the i-th projection hyperplane with probability proportional to |⟨ai, xk⟩ − bi| p , where 0 < p < ∞, for possible acceleration. It bridges the randomized Kaczmarz and greedy Kaczmarz by parameter p. Theoretically, we show its linear convergence rate in expectation with respect to the Bregman distance in the noiseless and noisy cases, which is at least as efficient as the randomized sparse Kaczmarz method. The superiority of the proposed method is demonstrated via a group of numerical experiments.

show abstract

An Implicit Representation and Iterative Solution of Randomly Sketched Linear Systems

Cited by 6 publications

References 27 publications

Selectable Set Randomized Kaczmarz

Selectable Set Randomized Kaczmarz

Randomized Block Adaptive Linear System Solvers

A weighted randomized sparse Kaczmarz method for solving linear systems

Contact Info

Product

Resources

About