On two-subspace randomized extended Kaczmarz method for solving large linear least-squares problems

Abstract

The randomized row method is a popular representative of the iterative algorithm because of its efficiency in solving the overdetermined and consistent systems of linear equations. In this paper, we present an extended randomized multiple row method to solve a given overdetermined and inconsistent linear system and analyze its computational complexities at each iteration. We prove that the proposed method can linearly converge in the mean square to the least-squares solution with a minimum Euclidean norm. Several numerical studies are presented to corroborate our theoretical findings. The real-world applications, such as image reconstruction and large noisy data fitting in computer-aided geometric design, are also presented for illustration purposes.

Greedy Randomized and Maximal Weighted Residual Kaczmarz Methods with Oblique Projection

“…In this subsection, the entries of our coefficient matrix are randomly generated in the interval [c, 1]. This set of experiments was also done in [30] and [46], and pointed out that when the value of c is close to 1, the rows of matrix A is closer to linear correlation. Theorem 3.1 and theorem 3.2 have shown the effectiveness of the GRKO method and the MWRKO method in this case.…”

On the randomized multiple row-action methods for solving linear least-squares problems

“…This kind of method is difficult to parallelize and needs to calculate the Moore-Penrose inverse. The second one is the pseudoinverse-free method, in which the projections of the previous iterate onto each row in a block are computed; see, e.g., the two-subspace randomized extended Kaczmarz method for a related fast solver [32] and the randomized extended average block Kaczmarz (REABK) for a survey on the latter [12].…”

“…The convergence analysis reveals that the resulting algorithm has a linear (sometimes referred to as exponential) convergence rate, which is bounded by explicit expression. As with many iterative methods in the row-action family [3,11,12,25,31,32,34], our method relies on the information of the right-hand side. The underlying idea is that first using an additional row-action iterate to output an approximation of b N and then utilizing another row-action iterate to an asymptotical consistent linear system.…”

Faster Deterministic Pseudoinverse-Free Block Extension of Motzkin Method for Large Consistent Linear Systems

“…In [23], the randomized block Kaczmarz (RBK) method was presented for solving linear least-squares problems, which was considered as a synthesis of the Elfving's blockiterative method and randomization. In [32], Wu presented the two-subspace REK method (TREK) for solving large-scale linear least-squares problems, which does not require any row or column paving. Recently, Niu and Zheng [25] developed a greed block Kaczmarz (GBK) method based on an approximate maximum distance (MD) greedy rule [26].…”