Information Theoretic Learning

Mean shift: An information theoretic perspective

“…Notice the argument of the Gaussian kernel which considers all possible pairs of samples. The idea of regarding the samples as information particles was first introduced by Príncipe et al and collaborators [12], [13] upon realizing that these samples interact with each other through laws that resembled the potential fields and their associated forces in physics.…”

“…The information potential and force experienced by particle x i ∈ X due to all particles of dataset Y is shown in (12) where F (x i | y j ) is the "cross" information force exerted by particle y j on particle x i . Similarly, one can easily derive the potential and force experienced by y i ∈ Y due to all particles of dataset X by simply interchanging M ↔ N and x ↔ y in (12).…”

“…Similarly, one can easily derive the potential and force experienced by y i ∈ Y due to all particles of dataset X by simply interchanging M ↔ N and x ↔ y in (12). Fig.…”

“…These ideas lie at the heart of information theoretic learning (ITL) [12]. By playing directly with pdf of the data and estimating the entropy non-parametrically, ITL effectively goes beyond the second order statistics.…”

Unsupervised decomposition of low-intensity low-dimensional multi-spectral fluorescent images for tumour demarcation

“…One alternation is to derive methods that are adaptive to the unknown source distributions. The representatives of such class of methods are kernel density methods (Xue et al, 2008;Principe and Xu, 1999;Principe et al, 2000). Main disadvantage of the kernel density based ICA methods is their computational complexity that is O(T 2 N 2 ) where T represents number of samples.…”