Matrix-free interior point method for compressed sensing problems

Abstract

We consider a class of optimization problems for sparse signal reconstruction which arise in the field of Compressed Sensing (CS). A plethora of approaches and solvers exist for such problems, for example GPSR, FPC AS, SPGL1, NestA, 1 s , PDCO to mention a few.CS applications lead to very well conditioned optimization problems and therefore can be solved easily by simple first-order methods. Interior point methods (IPMs) rely on the Newton method hence they use the second-order information. They have numerous advantageous features and one clear drawback: being the second-order approach they need to solve linear equations and this operation has (in the general dense case) an O(n 3 ) computational complexity. Attempts have been made to specialize IPMs to sparse reconstruction problems and they have led to interesting developments implemented in 1 s and PDCO softwares. We go a few steps further. First, we use the matrixfree IPM, an approach which redesigns IPM to avoid the need to explicitly formulate (and store) the Newton equation systems. Secondly, we exploit the 2 special features of the signal processing matrices within the matrix-free IPM. Two such features are of particular interest: an excellent conditioning of these matrices and the ability to perform inexpensive (low complexity) matrix-vector multiplications with them.Computational experience with large scale one-dimensional signals confirms that the new approach is efficient and offers an attractive alternative to other state-of-the-art solvers.

A Highly Efficient Semismooth Newton Augmented Lagrangian Method for Solving Lasso Problems

“…The first category consists of algorithms that use only the gradient information, for example, accelerated proximal gradient (APG) method [37,2], GPSR [16], SPGL1 [4], SpaRSA [50], FPC AS [49], and NESTA [3], to name only a few. Meanwhile, algorithms in the second category, including but not limited to mfIPM [18], SNF [36], BAS [6], SQA [7], OBA [26], FBS-Newton [51], utilize the second order information of the underlying problem in the algorithmic design to accelerate the convergence. Nearly all of these second order information based solvers rely on certain nondegeneracy assumptions to guarantee the non-singularity of the corresponding inner linear systems.…”

OSGA: a fast subgradient algorithm with optimal complexity

“…(However, a matrix-free interior point method by Fountoulakis et al [15] works well in some large compressed sensing problems).…”

A second-order method for convex1-regularized optimization with active-set prediction

“…On the other hand, in the extreme case |U F | = 1, the algorithm resembles a classical active-set method, which is not well-suited for large-scale problems. (2) while k = 0, 1, 2, · · · and stopping criterion not met do (3) Active-Set Identification: (4) and ζ k by (12).…”

“…(12) end if (13) end while (14) Set x k+1 = x k ISTA +ᾱ ·d k . (15) end while problem are representative of the performance of OBA on other functions f (including multi-class logistic regression, probit regression and LASSO) where similar trends are observed.…”

“…Email: nitishkeskar2012@u.northwestern.edu SpaRSA and FISTA [3,10,37], and proximal Newton methods that compute a step by minimizing a piecewise quadratic model of (1) using (for example) a coordinate descent iteration [5,16,19,23,26,29,32,38]. The proposed algorithm also differs from methods that solve (1) by reformulating it as a bound constrained problem [12,18,27,28,33,35], and from recent methods that are specifically designed for the case when f is a convex quadratic [9,30,34].…”