SOMNet: Unsupervised Feature Learning Networks for Image Classification

Abstract

We present here an unsupervised approach to learning suitable features for a deep learning framework applied to image classification. PCANet was introduced as a simple and efficient baseline for deep learning approaches which used cascaded principle component analysis (PCA) derived filter banks, as well as other simple image processing elements such as binary hashing and blockwise histograms. This was followed by DCTNet which used discrete cosine transform (DCT) filter banks as a learning-free alternative. In this paper we propose SOMNet which uses self-organizing map (SOM) based filters offering a non-orthogonal alternative to PCANet providing comparable performance. It is well established that SOM is a non-linear version of PCA but does not suffer from the same constraints. We also show that through the use of a simple trick in the binarization process results in a dramatic reduction in the dimension of the final feature vector, thus allowing the utilization of more filters which could lead to deeper and more complex structures in further work. We also demonstrate the results of a hybrid methodology that clusters generative Markov random fields (MRF) as filters which provides more diverse features in a data driven approach to deep learning. Index Terms-Self-organizing maps, unsupervised learning, Markov random fields, convolutional neural network, deep learning, handwritten digit recognition.

A Realist Evaluation of Case Management Models for People with Complex Health Conditions Using Novel Methods and Tools—What Works, for Whom, and under What Circumstances?

“…The SOMNet analysis uses data science techniques (data mining and machine learning) to learn associative relations (patterns) from multiple data sets and develop interactive visual data analytics of the pattern information to obtain insight into the data. Different to other machine learning modelling for classification, a SOM algorithm interprets data in a dataset to find similarities to each other, focusing on their relationship/association [45]. Such a clustering analysis can identify 'natural' patterns of the characteristics data around participants, their context, mechanism, and outcomes.…”

Deep Convolutional Self-Organizing Map Network for Robust Handwritten Digit Recognition

“…Instead of learning PCA filters, DCTNet [34] was developed to employ learning-free filters using Discrete Cosine Transform (DCT). On the same research direction, Hankins et al proposed SOMNet [13] to learn a set of nonorthogonal filters using self-organizing map (SOM). In their work, SOM is utilized to approximate the non-linear manifold of the input space through the discretized representation of neurons.…”

“…It is clear that, using 3D-SOM grid for DCSOM-2 features and 4D-SOM grid for DCSOM-1 features outperform state-of-the-art results in MNIST-rand and MNIST-img datasets, respectively. These 1.2 Deep belief network + linear SVM [41] 1.9 CDBN [42] 0.82 ConvNet [40] 0.53 DCTNet [34] 0.74 ScatNet-2 [12] 0.43 PCANet-2 [11] 0.66 LDANet-2 [11] 0.62 DSOM [23] 3.83 UDSOM [20] 1.06 CSOM (2D) [14] 0.81 SOMNet [13] 0.86 CR-MSOM [15] 0.97 DCSOM-1 (4D) 0.78 DCSOM-2 (4D) 0.57 results prove that features of DCSOM are more robust to noise than other methods when we carefully choose the appropriate dimension of SOM grid. The hard quantization of the SOM mapping highly improve the representation of the noisy patches as compared to other CNN architecture which use either supervised or unsupervised learning.…”

“…Information generated by these filters are combined to develop a new model for visual cortex [7]- [9]. Several unsupervised deep learning algorithms such as: K-means [10], PCANet [11], ScatNet [12] and SOMNet [13] are proposed to build a simple unsupervised deep network architecture.…”