On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty

Abstract

An explicit algorithm for the minimization of an ℓ 1 penalized least squares functional, with non-separable ℓ 1 term, is proposed. Each step in the iterative algorithm requires four matrix vector multiplications and a single simple projection on a convex set (or equivalently thresholding). Convergence is proven and a 1/N convergence rate is derived for the functional. In the special case where the matrix in the ℓ 1 term is the identity (or orthogonal), the algorithm reduces to the traditional iterative soft-thresholding algorithm. In the special case where the matrix in the quadratic term is the identity (or orthogonal), the algorithm reduces to a gradient projection algorithm for the dual problem.By replacing the projection with a simple proximity operator, other convex nonseparable penalties than those based on an ℓ 1 -norm can be handled as well.

Smoothing and Decomposition for Analysis Sparse Recovery

“…In the last example we also introduce a continuation technique to further speed up convergence of the smoothing-based method. We further compare SFISTA with the existing methods in [18], [20], [23] using MRI image reconstruction, and show its advantages. In this simulation, the entries in the m × n measurement matrix A were randomly generated according to a normal distribution.…”

“…We tested the algorithm on the Shepp Logan image from the previous subsection with the same setting, using SFISTA with µ f = 10 −4 λ −1 and standard SFISTA with µ = 10 −4 λ −1 . We implemented the generalized iterative soft-thresholding algorithm (GIST) from [20]. We also included an ADMM-based method, i.e.…”

“…A generalized iterative soft-thresholding algorithm was proposed in [20] to tackle this difficulty. However, this approach converges relatively slow as we will show in one of our numerical examples.…”

GISTA reconstructs faster with a restart strategy and even faster with a FISTA-like reconstruction

“…In this work, we propose to use the GIST algorithm [16] to reconstruct CT images with a TV penalty. GISTA gathers properties as: proven convergence, no subproblem to solve and reducing to ISTA when A is orthogonal.…”

“…In this scheme, the ISTA iteration is applied on a well-chosen linear combination of x n and x n−1 , instead of being applied on x n only [15]. Recently, Loris and Verhoeven (2011) [16] designed a generalization of ISTA (GISTA) having the following properties: (i) able to handle a non smooth and non invertible penalty term such as (4), (ii) having proven convergence, (iii) being explicit, ie not requiring the iterative solving of a subproblem at each iteration, and (iv) reducing to ISTA when A is orthogonal.…”