Self-organizing mixture networks for probability density estimation

Abstract

A self-organizing mixture network (SOMN) is derived for learning arbitrary density functions. The network minimizes the Kullback-Leibler information metric by means of stochastic approximation methods. The density functions are modeled as mixtures of parametric distributions. A mixture needs not to be homogenous, i.e., it can have different density profiles. The first layer of the network is similar to Kohonen's self-organizing map (SOM), but with the parameters of the component densities as the learning weights. The winning mechanism is based on maximum posterior probability, and updating of the weights is limited to a small neighborhood around the winner. The second layer accumulates the responses of these local nodes, weighted by the learned mixing parameters. The network possesses a simple structure and computational form, yet yields fast and robust convergence. The network has a generalization ability due to the relative entropy criterion used. Applications to density profile estimation and pattern classification are presented. The SOMN can also provide an insight to the role of neighborhood function used in the SOM.

Content-based image collection summarization and comparison using self-organizing maps

“…It also has been shown in [14] that the SOM can be regarded as a simplified Gaussian mixture estimator using a homoscedastic Gaussian mixture model. Trained with a large amount of input data, the feature map will form micro-clusters that can be treated as multivariate normal distributions.…”

The Self-Organizing Maps: Background, Theories, Extensions and Applications

“…Extensions along this direction continue to be a focus of research. Extension on probabilistic approaches which enhances the scope and capability of SOM include the Self-Organizing Mixture Network (SOMN) [128], Kernel-based topographic maps [106,107]; and the generic topographic mapping (GTM) [8]. There are many extensions developed in recent years -too many to completely list here.…”

“…If a spectrum consists of many components, then the SOMN described in Sect. 4.2 can be used to estimate the component profiles of the spectrum [128,129]. Re-sampling the observed spectrum will provide distribution data for training.…”

“…The self-organizing mixture network (SOMN) [128] extends and adapts the SOM to a mixture density model, in which each node characterizes a conditional probability distribution. The joint probability density of the data or the network output is described by a mixture distribution,…”

Expanding Self-Organizing Map for data visualization and cluster analysis

“…The regular grid can be used as a convenient visualization surface for showing different features of data such as the cluster tendencies of the data [8,12,16]. SOMs have been successfully applied in various engineering applications covering areas such as pattern recognition, full-text and image analysis, vector quantization, regression, financial data analysis, traveling salesman problem, and fault diagnosis [3,4,6,7,10,14,18].…”