Completion of Structured Low-Rank Matrices via Iteratively Reweighted Least Squares

Abstract

We propose a new Iteratively Reweighted Least Squares (IRLS) algorithm for the problem of completing or denoising low-rank matrices that are structured, e.g., that possess a Hankel, Toeplitz or block-Hankel/Toeplitz structure. The algorithm optimizes an objective based on a non-convex surrogate of the rank by solving a sequence of quadratic problems.Our strategy combines computational efficiency, as it operates on a lower dimensional generator space of the structured matrices, with high statistical accuracy which can be observed in experiments on hard estimation and completion tasks.Our experiments show that the proposed algorithm StrucHMIRLS exhibits an empirical recovery probability close to 1 from fewer samples than the state-of-the-art in a Hankel matrix completion task arising from the problem of spectral super-resolution of badly separated frequencies.Furthermore, we explain how the proposed algorithm for structured low-rank recovery can be used as preprocessing step for improved robustness in frequency or line spectrum estimation problems.

Hankel low-rank approximation and completion in time series analysis and forecasting: a brief review

“…Tighter relaxations were also proposed based on convex envelopes [21,6,4], [53,§VI] which take into account the particular cost function under consideration. Finally, several iteratively reweighted least squares methods based on surrogates for rank were proposed, which possess local convergence and theoretical guarantees [75,Ch.3], [74], and allow for a fast implementation [91].…”

Method for Sparse Representation of Complex Data Based on Overcomplete Basis, l1 Norm, and Neural MFNN-like Network

“…Below are the results of the numerical simulation of the operation of a neural network with various activation functions ( 18)- (20) under the influence of a single point source with an amplitude of . b 1 =1, an angle of arrival of θ = −22.5 • , and different phases of φ = 0 • , 72 • , and −18.9 • ; on the left, vertically, are the values of the phases used in modeling the sources 0 • , 72 • , and −18.9 • .…”

“…From the results obtained, it is clear that when the phase value φ = 0 • , that is, provided that the phase of the source and the phases of the basis functions (10) coincide, the use of all activation functions leads to a solution with only one non-zero basis coefficient, which corresponds to the amplitude and direction of the source, as shown in Figure 2a-c. However, if a discrepancy occurs between the phases of the source and the bases ( 18) and (20), it leads to errors in determining the direction and amplitude. In this case, the number of non-zero values of the basic coefficients significantly exceeds the number of sources, and among them, there are no corresponding directions to the source.…”

“…However, even in this case, with an increase in time by 10 times, the loss function stabilizes at a level less than 10 −14 without changing |X|), which indicates that the minimum has been reached, according to norm l 1 . Thus, like the case of using activation functions (18) and (20) and phase φ = 72 • , it is clear that the value of the loss function ( 3) turns out to be very small, but the parameters of the non-zero basis coefficients are very different from the parameters of the external source. In addition, the convergence process slows down.…”

“…An implementation option for the Algorithm 1 BP-IRLS (iteratively reweighted least squares) [20] is given below:…”