Sparse Approximate Solutions to Linear Systems

Abstract

The following problem is considered: given a matrix A in Rm'', (m rows and n columns), a vector b in Rm, and 6 > 0, compute a vector x satisfying IIAx bl[2 <_ 6 if such exists, such that x has the fewest number of non-zero entries over all such vectors. It is shown that the problem is NP-hard, but that the well-known greedy heuristic is good in that it computes a solution with at most [18 Opt(6/Z)llA + I1 ln(llbl12/6)] non-zero entries, where Opt(6/2) is the optimum number of nonzero entries at error 6/2, A is the matrix obtained by normalizing each column of A with respect to the L2 norm, and A + is its pseudo-inverse.

Optimal data collection for randomized control trials

“…It is known to be NP-hard (non-deterministic polynomial time hard) to find a solution to the inner optimization problem in (6) subject to the constraint that γ has m non-zero components, also called an m-term approximation, where m is the integer part of B/n in our problem. In other words, solving (6) directly is not feasible unless the dimension of covariates, M , is small (Natarajan, 1995;Davis, Mallat, and Avellaneda, 1997).…”

High-rate compression of ECG signals by an accuracy-driven sparsity model relying on natural basis

“…In practice, this problem is well-known to be NP-hard [18]. As mentioned in the introduction, there are various heuristics for finding an approximate solution to this problem.…”

Identifying small mean-reverting portfolios

“…This is a hard combinatorial problem, in fact, Natarajan (1995) shows that sparse generalized eigenvalue problems are equivalent to subset selection, which is NP-hard. We can't expect to get optimal solutions and we discuss below two efficient techniques to get good approximate solutions.…”