Big Data Clustering Using Improvised Fuzzy C-Means Clustering

Abstract

Clustering emerged as powerful mechanism to analyze the massive data generated by modern applications; the main aim of it is to categorize the data into clusters where objects are grouped into the particular category. However, there are various challenges while clustering the big data recently. Deep Learning has been powerful paradigm for big data analysis, this requires huge number of samples for training the model, which is time consuming and expensive. This can be avoided though fuzzy approach. In this research work, we design and develop an Improvised Fuzzy C-Means (IFCM)which comprises the encoder decoder Convolutional Neural Network (CNN) model and Fuzzy C-means (FCM) technique to enhance the clustering mechanism. Encoder decoder based CNN is used for learning feature and faster computation. In general, FCM, we introduce a function which measure the distance between the cluster center and instance which helps in achieving the better clustering and later we introduce Optimized Encoder Decoder (OED) CNN model for improvising the performance and for faster computation. Further in order to evaluate the proposed mechanism, three distinctive data types namely Modified National Institute of Standards and Technology (MNIST), fashion MNIST and United States Postal Service (USPS) are used, also evaluation is carried out by considering the performance metric like Accuracy, Adjusted Rand Index (ARI) and Normalized Mutual Information (NMI). Moreover, comparative analysis is carried out on each dataset and comparative analysis shows that IFCM outperforms the existing model.

Artificial Intelligence for Cluster Analysis: Case Study of Transport Companies in Czech Republic

“…A tool for analyzing the data of transport companies in the CR as well as the entire transport sector can be both traditional statistical and advanced research methods. (Rayala and Kalli 2021) state that currently, there are various methods (including advanced ones) for processing large volumes of data. The authors believe that deep learning could be a strong paradigm for analyzing large datasets; however, it requires many samples for model training, which is costly and time-consuming.…”

“…This can be avoided by using a fuzzy approach. (Rayala and Kalli 2021) proposed and developed an improvised Fuzzy C-means (IFCM), which includes the model of Convolutional Neural Network (CNN) and Fuzzy C-means (FCM) for improving the clustering mechanism. In addition, a comparative analysis was performed for each dataset, which showed that IFCM surpasses the existing model.…”

Adaptive Landmark-Based Spectral Clustering for Big Datasets

“…DFCM achieved a performance of 48.65% and 64.54% in terms of ARI and NMI, respectively, in the Fashion-MNIST dataset and 68.15% and 76.36% in terms of ARI and NMI, respectively, in the USPS dataset. To improve the clustering process, the encoder-decoder convolutional neural network (CNN) model and the FCM technique are combined in the improvised fuzzy c-means (IFCM) [29]. IFCM achieved a performance of 54.19% and 67.35% in terms of ARI and NMI, respectively, in the Fashion-MNIST dataset and 85% and 89% in terms of ARI and NMI, respectively, in the USPS dataset.…”

“…The autoencoder is regularized via the reconstruction mechanism. To improve the clustering process, improvised FCM [29] uses the encoder-decoder CNN model and FCM technique. The FCS method is an efficient fuzzy clustering method that estimates the fuzzy memberships of data using within and between-cluster distances.…”

“…The Rand index is a calculation of the closeness of two separate data clusters, and it ranges from 0 to 1. The higher the ARI value, the more accurate is that the received clustering model [29]. In addition to Figures 4 and 5, Tables 2 and 3 show the different sizes of subdatasets created from a big dataset using the USPS and Fashion-MNIST datasets, as well as the effect of subdataset size on the determination of the number of clusters (K).…”