Sparse Approximation of Multivariate Functions from Small Datasets Via Weighted Orthogonal Matching Pursuit

Abstract

We show the potential of greedy recovery strategies for the sparse approximation of multivariate functions from a small dataset of pointwise evaluations by considering an extension of the orthogonal matching pursuit to the setting of weighted sparsity. The proposed recovery strategy is based on a formal derivation of the greedy index selection rule. Numerical experiments show that the proposed weighted orthogonal matching pursuit algorithm is able to reach accuracy levels similar to those of weighted ℓ1 minimization programs while considerably improving the computational efficiency for small values of the sparsity level.

On Efficient Algorithms for Computing Near-Best Polynomial Approximations to High-Dimensional, Hilbert-Valued Functions from Limited Samples

“…Unfortunately, it is not clear how to extend these procedures to the weighted case with theoretical guarantees. Nonetheless, certain weighted greedy methods appear to work well in practice for sparse polynomial approximation [4].…”

On efficient algorithms for computing near-best polynomial approximations to high-dimensional, Hilbert-valued functions from limited samples

“…Unfortunately, it is not clear how to extend these procedures to the weighted case with theoretical guarantees. Nonetheless, certain weighted greedy methods appear to work well in practice for sparse polynomial approximation [5].…”

Towards optimal sampling for learning sparse approximation in high dimensions

“…Note that weighted sparsity and weighted 1 -minimization were first elaborated in [80], before further developments in [1,2,8,25]. Other works on incorporating weights into sparse polynomial approximation include [5,75,99]. As in the case of standard sparsity, successful recovery via (71) follows from a norm equivalence similar to (44): namely,…”