Randomized Sparse Block Kaczmarz as Randomized Dual Block-Coordinate Descent

Petra, Stefania

doi:10.1515/auom-2015-0052

Cited by 15 publications

(25 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Expected linear convergence for a randomized and smoothed Sparse Kaczmarz method was also shown in [33]. There the objective function (13) was replaced by…”

Section: Linear Convergence Of the Randomized Sparse Kaczmarz Methodsmentioning

confidence: 92%

Linear convergence of the randomized sparse Kaczmarz method

Schöpfer

Lorenz

2018

Math. Program.

View full text Add to dashboard Cite

The randomized version of the Kaczmarz method for the solution of linear systems is known to converge linearly in expectation. In this work we extend this result and show that the recently proposed Randomized Sparse Kaczmarz method for recovery of sparse solutions, as well as many variants, also converges linearly in expectation. The result is achieved in the framework of split feasibility problems and their solution by randomized Bregman projections with respect to strongly convex functions. To obtain the expected convergence rates we prove extensions of error bounds for projections. The convergence result is shown to hold in more general settings involving smooth convex functions, piecewise linear-quadratic functions and also the regularized nuclear norm, which is used in the area of low rank matrix problems. Numerical experiments indicate that the Randomized Sparse Kaczmarz method provides advantages over both the non-randomized and the non-sparse Kaczmarz methods for the solution of over-and under-determined linear systems.Definition 2.9. Let A ∈ R m×n , b ∈ R m , u ∈ R n and β ∈ R. By L(A, b) we denote the affine subspaceby H(u, β) the hyperplane H(u, β) := {x ∈ R n | u , x = β} , and by H ≤ (u, β) the half-space H ≤ (u, β) := {x ∈ R n | u , x ≤ β} .

show abstract

“…Expected linear convergence for a randomized and smoothed Sparse Kaczmarz method was also shown in [33]. There the objective function (13) was replaced by…”

Section: Linear Convergence Of the Randomized Sparse Kaczmarz Methodsmentioning

confidence: 92%

Linear convergence of the randomized sparse Kaczmarz method

Schöpfer

Lorenz

2018

Math. Program.

View full text Add to dashboard Cite

show abstract

“…, but in general this need not be the case. The following example was also used in [18] as a smoothed version of (4).…”

Section: Preliminariesmentioning

confidence: 99%

“…as sparsity promoting function, as was done in [18]. But this requires tuning the two parameters ε,τ instead of only λ.…”

Section: Of the Bregman Distance And Sincementioning

confidence: 99%

“…1 x k+1 = x k − ai , x k −bi ai 2 2 • a i , with initial value x 0 = 0. The Randomized Sparse Kaczmarz method [15,18,24] is a relatively new variant of the Kaczmarz method with almost the same low cost and storage requirements, and which has shown good performance in approximating sparse solutions of large consistent linear systems. It uses two variables x * k and x k and reads as…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Extended Randomized Kaczmarz Method for Sparse Least Squares and Impulsive Noise Problems

Schöpfer¹,

Lorenz²,

Tondji³

et al. 2022

Preprint

View full text Add to dashboard Cite

The Extended Randomized Kaczmarz method is a well known iterative scheme which can find the Moore-Penrose inverse solution of a possibly inconsistent linear system and requires only one additional column of the system matrix in each iteration in comparison with the standard randomized Kaczmarz method. Also, the Sparse Randomized Kaczmarz method has been shown to converge linearly to a sparse solution of a consistent linear system. Here, we combine both ideas and propose an Extended Sparse Randomized Kaczmarz method. We show linear expected convergence to a sparse least squares solution in the sense that an extended variant of the regularized basis pursuit problem is solved. Moreover, we generalize the additional step in the method and prove convergence to a more abstract optimization problem. We demonstrate numerically that our method can find sparse least squares solutions of real and complex systems if the noise is concentrated in the com-The work of L.T. and D.L. has been supported by the ITN-ETN project TraDE-OPT funded by the European Union's Horizon 2020 research and innovation programme under the Marie Sk lodowska-Curie grant agreement No 861137. This work represents only the author's view and the European Commission is not responsible for any use that may be made of the information it contains.

show abstract

“…where A i k : is the i k -row of A randomly selected at iteration k and b i k is the i k -entry of b. Recently, remarkable progress of the Kaczmarz method has been made; see for example [2][3][4][5][6][7][8][9][10][11][12][13].…”

Section: Introductionmentioning

confidence: 99%

Adaptively Sketched Bregman Projection Methods for Linear Systems

Yuan,

Zhang,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

The sketch-and-project, as a general archetypal algorithm for solving linear systems, unifies a variety of randomized iterative methods such as the randomized Kaczmarz and randomized coordinate descent. However, since it aims to find a least-norm solution from a linear system, the randomized sparse Kaczmarz can not be included. This motivates us to propose a more general framework, called sketched Bregman projection (SBP) method, in which we are able to find solutions with certain structures from linear systems. To generalize the concept of adaptive sampling to the SBP method, we show how the progress, measured by Bregman distance, of single step depends directly on a sketched loss function. Theoretically, we provide detailed global convergence results for the SBP method with different adaptive sampling rules. At last, for the (sparse) Kaczmarz methods, a group of numerical simulations are tested, with which we verify that the methods utilizing sampling Kaczmarz-Motzkin rule demands the fewest computational costs to achieve a given error bound comparing to the corresponding methods with other sampling rules.

show abstract

Randomized Sparse Block Kaczmarz as Randomized Dual Block-Coordinate Descent

Cited by 15 publications

References 19 publications

Linear convergence of the randomized sparse Kaczmarz method

Linear convergence of the randomized sparse Kaczmarz method

Extended Randomized Kaczmarz Method for Sparse Least Squares and Impulsive Noise Problems

Adaptively Sketched Bregman Projection Methods for Linear Systems

Contact Info

Product

Resources

About