Linear and nonlinear ICA based on mutual information

Almeida, Luís B.

doi:10.1109/asspcc.2000.882457

Cited by 17 publications

(23 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The mutual information is related to the marginal entropies and the joint entropy of these random variables through (Cover and Thomas, 1991) Minimization of output mutual information is "the canonical contrast for source separation" as Cardoso states (Cardoso and Souloumiac, 1993). Many researchers agree with this comment (Yang and Amari, 1997;Hyvarinen, 1999a;Almeida, 2000). However, three of the most well known methods for ICA, namely JADE (Cardoso and Souloumniac, 1993), Infomax (Bell and Sejnowski, 1995), and FastICA (Hyvarinen, 1999b), use the diagonalization of cumulant matrices, maximization of output entropy, and fourth order-cumulants, respectively.…”

Section: Introductionmentioning

confidence: 99%

Minimax Mutual Information Approach for Independent Component Analysis

et al. 2004

View full text Add to dashboard Cite

Minimum output mutual information is regarded as a natural criterion for independent component analysis (ICA) and is used as the performance measure in many ICA algorithms. Two common approaches in information theoretic ICA algorithms are minimum mutual information and maximum output entropy approaches. In the former approach, we substitute some form of probability density function (pdf) estimate into the mutual information expression, and in the latter we incorporate the source pdf assumption in the algorithm through the use of nonlinearities matched to the corresponding cumulative density functions (cdf).Alternative solutions to ICA utilize higher order cumulant-based optimization criteria, which are related to either one of these approaches through truncated series approximations for densities. In this paper, we propose a new ICA algorithm motivated by the Maximum Entropy Principle (for estimating signal distributions).The optimality criterion is the minimum output mutual information, where the estimated pdfs are from the exponential family, and are approximate solutions to a constrained entropy maximization problem. This approach yields an upper bound for the actual mutual information of the output signals, hence the name Minimax Mutual Information ICA algorithm. In addition, we demonstrate that for a specific selection of the constraint functions in the maximum entropy density estimation procedure, the algorithm relates strongly to ICA methods using higher order cumulants.

show abstract

Section: Introductionmentioning

confidence: 99%

Minimax Mutual Information Approach for Independent Component Analysis

et al. 2004

View full text Add to dashboard Cite

show abstract

“…Recently, multi-layer perceptron (MLP) neural networks [47] have been used in [114,8,7] for estimating the nonlinear separating transform . For justifying this choice, in addition to the universal approximation property of MLP, Almeida claims that restricting the target transforms to the set of smooth transforms 6 generated by an MLP provides regularized solutions which ensures that nonlinear ICA leads to source separation.…”

Section: Smooth Transformsmentioning

confidence: 99%

“…Marques and Almeida [75,8] generalized the Infomax principle [21] to nonlinear mixtures. To this purpose, they propose a separation structure realized using a multi-layer perceptron (MLP) neural network.…”

Section: Constrained Mlp-like Structuresmentioning

confidence: 99%

Nonlinear mixtures

Jutten¹,

Babaie‐Zadeh²,

Karhunen³

2010

Handbook of Blind Source Separation

View full text Add to dashboard Cite

“…The latter class, which interests us most in this paper, has already been the subject of several publications (e.g. [3,4,5,6,7,8,9,10,11,12]).…”

Section: Introductionmentioning

confidence: 99%

10.1162/jmlr.2003.4.7-8.1297

Almeida

2000

CrossRef Listing of Deleted DOIs

View full text Add to dashboard Cite

MISEP is a method for linear and nonlinear ICA, that is able to handle a large variety of situations. It is an extension of the well known INFOMAX method, in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs. The method can therefore separate linear and nonlinear mixtures of components with a wide range of statistical distributions. This paper presents the basis of the MISEP method, as well as experimental results obtained with it. The results illustrate the applicability of the method to various situations, and show that, although the nonlinear blind separation problem is ill-posed, use of regularization allows the problem to be solved when the nonlinear mixture is relatively smooth.

show abstract

Linear and nonlinear ICA based on mutual information

Cited by 17 publications

References 20 publications

Minimax Mutual Information Approach for Independent Component Analysis

Minimax Mutual Information Approach for Independent Component Analysis

Nonlinear mixtures

10.1162/jmlr.2003.4.7-8.1297

Contact Info

Product

Resources

About