Efficient Sparse Representation for Learning With High-Dimensional Data

Abstract

Due to the capability of effectively learning intrinsic structures from high-dimensional data, techniques based on sparse representation have begun to display an impressive impact in several fields, such as image processing, computer vision and pattern recognition. Learning sparse representations is often computationally expensive due to the iterative computations needed to solve convex optimization problems in which the number of iterations is unknown before convergence. Moreover, most sparse representation algorithms focus only on determining the final sparse representation results and ignore the changes in the sparsity ratio during iterative computations. In this paper, two algorithms are proposed to learn sparse representations based on locality-constrained linear representation learning with probabilistic simplex constraints. Specifically, the first algorithm, called approximated local linear representation (ALLR), obtains a closed-form solution from individual locality-constrained sparse representations. The second algorithm, called approximated local linear representation with symmetric constraints (ALLRSC), further obtains all symmetric sparse representation results with a limited number of computations; notably, the sparsity and convergence of sparse representations can be guaranteed based on theoretical analysis. The steady decline in the sparsity ratio during iterative computations is a critical factor in practical applications. Experimental results based on public datasets demonstrate that the proposed algorithms perform better than several state-of-the-art algorithms for learning with high-dimensional data.

Two-Stage Sparse Representation Clustering for Dynamic Data Streams

“…Sparse representation is an extremely successful technique for representing high-dimensional data [33], [34]. Let the dictionary D = [d 1 , d 2 , ..., d n ] ∈ R d×n be a set of n vectors, where each column of the set is of length d. Given a vector x ∈ R d , its sparsest representation over the dictionary D can be approximately represented by a sparse linear combination of only a few columns of D. Specifically, a sparse representation vector z ∈ R n of x can be calculated via the following l 0 -norm minimization optimization problem:…”

scIALM: A method for sparse scRNA-seq expression matrix imputation using the Inexact Augmented Lagrange Multiplier with low error

“…It means that the original data can be reconstructed with high probability using a small number of observations [33] . The three core issues of compressed sensing theory are sparse representation of signals [34] , [35] , design of observation matrices, and reconstruction algorithms. In many practical problems, the data are two-dimensional matrices.…”