Non-Negative Orthogonal Greedy Algorithms

Abstract

Orthogonal greedy algorithms are popular sparse signal reconstruction algorithms. Their principle is to select atoms one by one. A series of unconstrained least-squares subproblems of gradually increasing size is solved to compute the approximation coefficients, which is efficiently performed using a fast recursive update scheme. When dealing with nonnegative sparse signal reconstruction, a series of non-negative least-squares subproblems have to be solved. Fast implementation becomes tricky since each subproblem does not have a closedform solution anymore. Recently, non-negative extensions of the classical orthogonal matching pursuit and orthogonal least squares algorithms were proposed, using slow (i.e., non-recursive) or recursive but inexact implementations. In this paper, we revisit these algorithms in a unified way. We define a class of non-negative orthogonal algorithms and exhibit their structural properties. We propose a fast and exact implementation based on the active-set resolution of non-negative least-squares and exploiting warm start initializations. The algorithms are assessed in terms of accuracy and computational complexity for a sparse spike deconvolution problem. We also present an application to near-infrared spectra decomposition.

Exact Sparse Nonnegative Least Squares

“…Popular examples are orthogonal matching pursuit (OMP) [13] and orthogonal least squares (OLS) [14]. Nonnegative variants of these algorithms have been proposed; see [15] and the references therein. They aim to solve (2) heuristically, but theoretical recovery guarantees are similarly limited.…”

“…• Nonnegative OMP (NNOMP) and Nonnegative OLS (NNOLS) [15] for which the Matlab codes are provided by the authors. Synthetic test cases are built by generating a random matrix A ∈ R m×n + and a random k-sparse vector xtrue ∈ R n + , computing b = Axtrue, and trying to find again xtrue with A, b and k as parameters of the sparse NNLS algorithm.…”

A Partially Collapsed Gibbs Sampler for Unsupervised Nonnegative Sparse Signal Restoration

“…For example, the 1 relaxation can be easily extended to the nonnegative setting [5,6]. Nonnegative extensions of greedy algorithms have been also introduced [7,8].…”

K-step analysis of orthogonal greedy algorithms for non-negative sparse representations

“…This yields an increase of computation time since NNLS subproblems do not have closed-form solutions, and an iterative subroutine is needed. In [27], following the early work of [36], we proposed fully recursive implementations. We showed that non-negative greedy algorithms yield accurate empirical results and that their computation cost is of the same order of magnitude as those of Oxx for moderate size problems.…”

“…Non-negative OMP was first introduced by Bruckstein et al [5] under the name OMP, and then renamed NNOMP in [36] (see also [21,27]). It relies on the repeated maximization of the positive inner product between the residual vector and the dictionary atoms, followed by the resolution of an NNLS problem.…”