Improving Self-Organizing Maps with Unsupervised Feature Extraction

Khacef, Lyes; Rodriguez, Laurent; Miramond, Benoît

doi:10.1007/978-3-030-63833-7_40

Cited by 12 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notice that our model has used the datasets in a raw form, without any feature extractor, which might radically enhance the quality of the vectors representing the data. The addition of feature extractors as input of the neural maps increases the algorithm's accuracy, as we showed in [32], [76].…”

Section: A Simulation Resultsmentioning

confidence: 81%

“…Lastly, it should be noted that the problem addressed by our framework does not correspond to a typical SSL problem, since the existing labels are available (or provided) only after the training. So we consider a special type of SSL problem, called in the literature the Post-Labeled Unsupervised Learning problem [32], [76].…”

Section: Application Of Resom To Classification Tasksmentioning

confidence: 99%

See 1 more Smart Citation

A unified software/hardware scalable architecture for brain-inspired computing based on self-organizing neural models

Muliukov,

Rodriguez,

Miramond

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The field of artificial intelligence has significantly advanced over the past decades, inspired by discoveries from the fields of biology and neuroscience. The idea of this work is inspired by the process of self-organization of cortical areas in the human brain from both afferent and lateral/internal connections. In this work, we develop an original brain-inspired neural model associating Self-Organizing Maps (SOM) and Hebbian learning in the Reentrant SOM (ReSOM) model. The framework is applied to multimodal classification problems. Compared to existing methods based on unsupervised learning with post-labeling, the model enhances the state-of-the-art results. This work also demonstrates the distributed and scalable nature of the model through both simulation results and hardware execution on a dedicated FPGA-based platform named SCALP (Self-configurable 3D Cellular Adaptive Platform). SCALP boards can be interconnected in a modular way to support the structure of the neural model. Such a unified software and hardware approach enables the processing to be scaled and allows information from several modalities to be merged dynamically. The deployment on hardware boards provides performance results of parallel execution on several devices, with the communication between each board through dedicated serial links. The proposed unified architecture, composed of the ReSOM model and the SCALP hardware platform, demonstrates a significant increase in accuracy thanks to multimodal association, and a good trade-off between latency and power consumption compared to a centralized GPU implementation.

show abstract

Section: A Simulation Resultsmentioning

confidence: 81%

Section: Application Of Resom To Classification Tasksmentioning

confidence: 99%

A unified software/hardware scalable architecture for brain-inspired computing based on self-organizing neural models

Muliukov,

Rodriguez,

Miramond

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Shallow SubTab is trained and evaluated under the same conditions as the deeper ones. As shown in Table 3, shallow SubTab significantly improves results in MNIST and TCGA, placing our model performance on par with CNN-based SOTA models [20,19,25,22,32] as shown in Figure 4b. Obesity is the only dataset which exploits the deeper architecture.…”

Section: Ablation Studymentioning

confidence: 72%

SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning

Ucar¹,

Hajiramezanali²,

Edwards³

2021

Preprint

View full text Add to dashboard Cite

Self-supervised learning has been shown to be very effective in learning useful representations, and yet much of the success is achieved in data types such as images, audio, and text. The success is mainly enabled by taking advantage of spatial, temporal, or semantic structure in the data through augmentation. However, such structure may not exist in tabular datasets commonly used in fields such as healthcare, making it difficult to design an effective augmentation method, and hindering a similar progress in tabular data setting. In this paper, we introduce a new framework, Subsetting features of Tabular data (SubTab), that turns the task of learning from tabular data into a multi-view representation learning problem by dividing the input features to multiple subsets. We argue that reconstructing the data from the subset of its features rather than its corrupted version in an autoencoder setting can better capture its underlying latent representation. In this framework, the joint representation can be expressed as the aggregate of latent variables of the subsets at test time, which we refer to as collaborative inference. Our experiments show that the SubTab achieves the state of the art (SOTA) performance of 98.31% on MNIST in tabular setting, on par with CNN-based SOTA models, and surpasses existing baselines on three other real-world datasets by a significant margin.

show abstract

“…To accurately cluster such a dataset, the scattered subclusters must be merged into one cluster. Khacef et al (2019) and Khacef, Rodriguez & Miramond (2020) have accurately clustered MNIST using only six hundred labeled data points. Their proposed method projects input data onto a 2-dimensional feature map using SOM and then labels reference vectors using post-labeled unsupervised learning.…”

Section: Discussionmentioning

confidence: 99%

Approximate spectral clustering using both reference vectors and topology of the network generated by growing neural gas

Fujita

2021

PeerJ Computer Science

View full text Add to dashboard Cite

Spectral clustering (SC) is one of the most popular clustering methods and often outperforms traditional clustering methods. SC uses the eigenvectors of a Laplacian matrix calculated from a similarity matrix of a dataset. SC has serious drawbacks: the significant increases in the time complexity derived from the computation of eigenvectors and the memory space complexity to store the similarity matrix. To address the issues, I develop a new approximate spectral clustering using the network generated by growing neural gas (GNG), called ASC with GNG in this study. ASC with GNG uses not only reference vectors for vector quantization but also the topology of the network for extraction of the topological relationship between data points in a dataset. ASC with GNG calculates the similarity matrix from both the reference vectors and the topology of the network generated by GNG. Using the network generated from a dataset by GNG, ASC with GNG achieves to reduce the computational and space complexities and improve clustering quality. In this study, I demonstrate that ASC with GNG effectively reduces the computational time. Moreover, this study shows that ASC with GNG provides equal to or better clustering performance than SC.

show abstract

Improving Self-Organizing Maps with Unsupervised Feature Extraction

Cited by 12 publications

References 26 publications

A unified software/hardware scalable architecture for brain-inspired computing based on self-organizing neural models

A unified software/hardware scalable architecture for brain-inspired computing based on self-organizing neural models

SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning

Approximate spectral clustering using both reference vectors and topology of the network generated by growing neural gas

Contact Info

Product

Resources

About