Simple mixture model for sparse overcomplete ICA

Abstract

The use of mixture of Gaussians (MoGs) for noisy and overcomplete independent component analysis (ICA) when the source distributions are very sparse is explored. The sparsity model can often be justified if an appropriate transform, such as the modified discrete cosine transform, is used. Given the sparsity assumption, a number of simplifying approximations are introduced to the observation density that avoid the exponential growth of mixture components. An efficient clustering algorithm is derived whose complexity grows linearly with the number of sources and it is shown that it is capable of performing reasonable separation.

On the Cramér-Rao Bound for Estimating the Mixing Matrix in Noisy Sparse Component Analysis

“…3 Moreover, the parameter , which determines the width of the Cauchy distribution, is assumed to be small enough compared to the distance between the centers of the Cauchy distributions. 4 A typical figure of these two Cauchy distributions is depicted in Fig. 1.…”

Bayesian Approach for Blind Separation of Underdetermined Mixtures of Sparse Sources

“…(3) is a very convenient property which will help us deriving posterior distributions of the parameters in the implementation of the Gibbs sampler. The Student t can be interpreted as an infinite sum of Gaussians, which contrasts with the finite sums of Gaussians used in [3,4]. In the following, we note…”

“…More specifically, in [3,4] the coefficients of the representations of the sources in the dictionary are given a discrete mixture a Gaussian distributions with 2 or 3 states (one Gaussian with very small variance, the other(s) with big variance) and a probabilistic framework is presented for the estimation of the mixing matrix and the sources. In particular, in [4], the authors use EM optimisation and present results with speech signals decomposed on a MDCT orthogonal basis [5].…”

“…In particular, in [4], the authors use EM optimisation and present results with speech signals decomposed on a MDCT orthogonal basis [5]. The use of an orthogonal basis provides equivalence between representations in the time domain and transformed domain, and separation can be simply performed in the transformed domain instead of the time domain.…”

Blind Spectral-GMM Estimation for Underdetermined Instantaneous Audio Source Separation

“…In the audio domain, sparse prior distributions are usually a Laplacian [1], a generalized Gaussian [2], a Student-t [3], or a mixture of two Gaussians [4].…”