Analysis of Sparse Representation and Blind Source Separation

Abstract

In this letter, we analyze a two-stage cluster-then-l(1)-optimization approach for sparse representation of a data matrix, which is also a promising approach for blind source separation (BSS) in which fewer sensors than sources are present. First, sparse representation (factorization) of a data matrix is discussed. For a given overcomplete basis matrix, the corresponding sparse solution (coefficient matrix) with minimum l(1) norm is unique with probability one, which can be obtained using a standard linear programming algorithm. The equivalence of the l(1)-norm solution and the l(0)-norm solution is also analyzed according to a probabilistic framework. If the obtained l(1)-norm solution is sufficiently sparse, then it is equal to the l(0)-norm solution with a high probability. Furthermore, the l(1)- norm solution is robust to noise, but the l(0)-norm solution is not, showing that the l(1)-norm is a good sparsity measure. These results can be used as a recoverability analysis of BSS, as discussed. The basis matrix in this article is estimated using a clustering algorithm followed by normalization, in which the matrix columns are the cluster centers of normalized data column vectors. Zibulevsky, Pearlmutter, Boll, and Kisilev (2000) used this kind of two-stage approach in underdetermined BSS. Our recoverability analysis shows that this approach can deal with the situation in which the sources are overlapped to some degree in the analyzed domain and with the case in which the source number is unknown. It is also robust to additive noise and estimation error in the mixing matrix. Finally, four simulation examples and an EEG data analysis example are presented to illustrate the algorithm's utility and demonstrate its performance.

Extraction of multiple pure component 1H and 13C NMR spectra from two mixtures: Novel solution obtained by sparse component analysis-based blind decomposition

“…The reason is that even when A is known solution of the linear system of equations (1) or (7) is not unique, because there are more unknowns (M) than equations (N). If pure components are (M-N+1)-sparse, a unique solution is obtained at the minimum of the 1  norm of s, [24][25][26][27][30][31][32][33][34]. We could formulate linear programming based solution in the time-scale basis (7).…”

“…Thus, a BSS method capable to extract pure components from reduced number of mixtures (that is less than the number of pure components) appears to be of great importance. This leads to underdetermined BSS (uBSS) problem that is not solvable under standard ICA assumptions, [24][25][26][27].…”

“…Here, we propose a sparse component analysis (SCA) approach to solve related uBSS problem by having at disposal two mixtures only, [24][25][26][27]. It combines geometric approach known as clustering in wavelet domain to estimate concentration or mixing matrix and linear programming in Fourier/frequency domain to estimate pure components.…”

“…When N<M the BSS problem is underdetermined because there are less measured mixtures N available than unknown pure components M. In order to solve the uBSS problem additional a priori information about the pure components must be available, [24][25][26][27]. A priori information that is used most often is sparseness of the source signals in suitably chosen basis, [24][25][26][27]. [8] by the fact that it maximally exploits redundancy of the linear data model (1) in the chosen basis.…”

“…The SCA method in [8] solves BSS problem by finding de-mixing matrix W by minimizing cost function that measures sparseness of the sources, however it still requires N=M. On the other side the SCA approach used here, and referred in [24][25][26][27], breaks down BSS problem into two separate problems: estimation of the mixing or concentration matrix A using geometric concept known as data clustering [24][25][26][27][28][29], and estimation of the magnitude spectra of the pure components (based on estimated A) by solving resulting underdetermined system of linear equations through linear programming [24,25,30,31], 1  -regularized least square problem [32,33] or 2  -regularized linear problem, [34]. In the case of the NMR spectroscopy it is customary to assume that Fourier basis yields sparse representation, however wavelet basis with properly chosen wavelet function can yield even sparser representation.…”