Surrounding the solution of a Linear System of Equations from all sides

Abstract

Suppose A ∈ R n×n is invertible and we are looking for the solution of Ax = b. Given an initial guess x 1 ∈ R, we show that by reflecting through hyperplanes generated by the rows of A, we can generate an infinite sequence (x k ) ∞ k=1 such that all elements have the same distance to the solution, i.e.If the hyperplanes are chosen at random, averages over the sequence converge andThe bound does not depend on the dimension of the matrix. This introduces a purely geometric way of attacking the problem: are there fast ways of estimating the location of the center of a sphere from knowing many points on the sphere? Our convergence rate (coinciding with that of the Random Kaczmarz method) comes from averaging, can one do better?

A deterministic Kaczmarz algorithm for solving linear systems

“…So based on Theorem 2, it totally costs O(n 2 /ǫ 2 + n 2 /ǫ min l |θ l |) operations to obtain an approximation to the solution. This result is close to that of [7], which uses O(n 2 A −1 2 /ǫ 2 ) operations in total (here we assumed that each row has unit norm). In theory, in terms of ǫ, our result is worse than the randomized Kaczmarz method [8], which only uses O(n 2 A −1 2 log(1/ǫ 2 )) operations in expectation.…”

“…In [7], in the k-th step of iteration, i k is chosen with probability A i k 2 / A 2 F , which is 1/n under our above assumption. In principle, the above convergence result (7) is worse than the randomized Kaczmarz method [8], which has…”

“…Here m = O(s 2 κ log(1/ǫ 2 )). In this work, we go one step further than [7]. We will not choose i k randomly in the procedure (1), instead we choose it in sequence, i.e, i k = k mod n. This makes the algorithm deterministic.…”

“….} to approximate the solution, i.e., the method introduced in [7], see equation (7). The blue line uses a similar idea except that i k is chosen in order.…”

Randomized Douglas-Rachford method for linear systems: Improved accuracy and efficiency

“…They generalized the study in [35] and split the total error into the low-and high-frequency solution spaces. Furthermore, the reflection Kaczmarz method was studied by Steinerberger in [58]. The reflection Kaczmarz method constructs x k+1 by…”