BELMKN: Bayesian Extreme Learning Machines Kohonen Network

Abstract

This paper proposes the Bayesian Extreme Learning Machine Kohonen Network (BELMKN) framework to solve the clustering problem. The BELMKN framework uses three levels in processing nonlinearly separable datasets to obtain efficient clustering in terms of accuracy. In the first level, the Extreme Learning Machine (ELM)-based feature learning approach captures the nonlinearity in the data distribution by mapping it onto a d-dimensional space. In the second level, ELM-based feature extracted data is used as an input for Bayesian Information Criterion (BIC) to predict the number of clusters termed as a cluster prediction. In the final level, feature-extracted data along with the cluster prediction is passed to the Kohonen Network to obtain improved clustering accuracy. The main advantage of the proposed method is to overcome the problem of having a priori identifiers or class labels for the data; it is difficult to obtain labels in most of the cases for the real world datasets. The BELMKN framework is applied to 3 synthetic datasets and 10 benchmark datasets from the UCI machine learning repository and compared with the state-of-the-art clustering methods. The experimental results show that the proposed BELMKN-based clustering outperforms other clustering algorithms for the majority of the datasets. Hence, the BELMKN framework can be used to improve the clustering accuracy of the nonlinearly separable datasets.

A New Probabilistic Output Constrained Optimization Extreme Learning Machine

“…CPP-POELM preserves the merits of PO-ELM where it still engages fast and one pass learning in which output weights are computed straightaway that eliminates exhaustive iterative learning procedures. Unlike other existing Bayesian implementations in ELM literatures [10]- [13] which showcase that heavy iterative operations are still required to learn the output weights are however deemed slightly deviated from the underlying principles of fast learning ELM. Other superiorities of CPP-POELM include…”

“…Here, in order to improve the stability of learning, I C is introduced in the Equation (9) as stated by Senthilnath et al [13]. I is the identity matrix of same size with H T H, C is a user-specific parameter.…”

“…Another recent published work on Bayesian Extreme Learning Machines Kohonen Network (BELMKN) in 2018 by Senthilnath et al [13] is to solve clustering problem with 3 phases of unsupervised learning without Bayesian probability concept which is different from our strategy of implementation of Bayesian framework in ELM. The BELMKN starts with ELM feature learning in phase-1, and then uses Bayesian Information Criterion (BIC) to determine number of cluster based on the training samples that has been extracted and processed by ELM in phase-2, and then employs Kohonen Network to compress the extracted data to improve clustering.…”

“…Owing to this limitation, Platt uses a post processing method to map non-probabilistic outputs to probabilistic outputs by employing a trained parametric model. Note that the additional sigmoid fitting in the post-processing stage required exhaustive training and optimization of parameters A and B in Equation (12) and (13), in accordance with an acceptable test error amount. In contrast, the proposed method eliminates additional post-processing step to convert output into probabilities.…”

Advanced Artificial Neural Networks

“…Senthilnath et al [1] built a Bayesian extreme learning machine Kohonen network (BELMKN) to solve the clustering problem of nonlinearly separable datasets. The BELMKN had three layers.…”