Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance

Abstract

Abstract-Sparse signal models are used in many signal processing applications. The task of estimating the sparsest coefficient vector in these models is a combinatorial problem and efficient, often sub-optimal strategies have to be used. Fortunately, under certain conditions on the model, several algorithms could be shown to efficiently calculate near optimal solutions. In this paper, we study one of these methods, the so called Iterative Hard Thresholding algorithm. We are here interested in the application of this method to real world problems, in which it is not known in general, whether the conditions used in the performance guarantees are satisfied or not. We suggest a simple modification to the algorithm that guarantees the convergence of the method, even in a regime in which the theoretical condition is not satisfied. With this modification, empirical evidence suggests that the algorithm is faster than many other state of the art approaches whilst showing similar performance. What is more, the modified algorithm retains theoretical performance guarantees similar to the original algorithm.

Recipes on hard thresholding methods

“…We observe that memory acceleration does not degrade the signal reconstruction performance compared to equivalent zeromemory schemes. As a side remark, we note that 1-ALPS(0) behaves better than AIHT [8] and NIHT [7] algorithms in terms of phase transition performance.…”

“…Unfortunately, most of the above problem assumptions are not naturally met; the authors in [7] provide an intuitive example where IHT algorithm behaves differently under various scalings of the sensing matrix Φ. Violation of these configuration details usually lead to unpredictable signal recovery performance of hard thresholding methods.…”

“…In this experiment, we compare the signal recovery behaviour of 0-ALPS(4) algorithm using our adaptive step size selection and HTP algorithm [11] with NIHT adaptive µi selection [7]. Here, we assume N = 1000.…”

A Review on Image Reconstruction Using Compressed Sensing Algorithms: OMP, CoSaMP and NIHT

“…The fact that RIC is sensitive to rescaling of rendering instability to this algorithm. Hence instead on iterative thresholding its normalized version [22] is utilized that guarantees stability. A new factor μ is added and chosen adaptively, instead of constraining to …”

Phase Retrieval via Iterative Hard Thresholding Backtracking Algorithm

“…In 2008, with the help of surrogate objective function to transform the traditional minimization problem, and a sufficient condition for the convergence of IHT to a fixed point is given [9]. In 2010, Blumensath and Davies improved fixed step size into variable step size, making IHT be generalized to Normalised IHT(NIHT) [10]. In the following year, based on the idea of backtracking, BIHT is proposed to solve the problem of compressed sensing [11].…”