A Novel Clustering-Based Sampling Approach for Minimum Sample Set in Big Data Environment

Abstract

The data are rapidly expanding nowadays, which makes it very difficult to analyze valuable information from big data. Most of the existing data mining algorithms deal with big data problems at large time and space costs. This paper focuses on the sampling problem of big data and puts forward an efficient heuristic Cluster Sampling Arithmetic, called CSA. Many of the former researchers adopted random method to extract early sample set from the original data and then made a variety of different processing of the sample in order to obtain the corresponding minimum sample set, which is regarded as a representation of the original big data set. However, the final processing results of big data will be severely affected by the random sampling process at the beginning, resulting in lower comprehensiveness and quality of the final data results and longer processing time. Based on this view, CSA introduces the idea of clustering to obtain minimum sample set of big data, which is in contrast to the random sampling method in the current literature. CSA makes cluster analysis of the original data set and selects the center of each class as centralized members of the minimum sample set. It aims at ensuring that the sample distribution accords with the characteristics of the original data, guarantees the data integrity and reduces the processing time. The max–min distance means that the pattern recognition has been integrated into the clustering process in order to get the clustering center and prevent algorithm from local optimum. The final experimental results show that, compared with the existing work, CSA algorithm can efficiently reflect the characteristics of the original data and reduce the time of data processing. The obtained minimum sample set has also achieved good effects in the classification algorithm.

Driving Risk Classification Methodology for Intelligent Drive in Real Traffic Event

“…In the big data era, popular technologies like cloud platforms, cloud computing and data warehouse emerge as per today's requirement. 10,28 In a word, it is possible to build a clear correlation among driving data, driving behaviors and driving safety with an e®ective utilization and integration of various vehicles data through data mining, which also provides technological services in driving safety for intelligent vehicles in the future. 25,26 Meanwhile, the Society of Automotive Engineers (SAE) has divided vehicle automation into six levels.…”

Automatic Determination of Clustering Centers for “Clustering by Fast Search and Find of Density Peaks”

“…Because comprehensive metrics c and distances between potential clustering centers are always related [41], they can be integrated to automatically identifying clustering centers.…”

“…We improved the algorithm proposed by Zhao [41,42] to recognize the clustering centers. Discriminant distance φ T i is defined as…”

“…where δ T i is the minimum density-based distance according to c ,sorted in descending order, and i is the number of determinate clustering centers. e discriminant condition of clustering centers is defined as [41] Dm…”

HaloDPC: An Improved Recognition Method on Halo Node for Density Peak Clustering Algorithm

“…Clustering has been applied in a wide variety of¯elds, ranging from engineering (machine learning, arti¯cial intelligence, pattern recognition, mechanical engineering, electrical engineering), 2 computer sciences (web mining, spatial database analysis, textual document collection, image segmentation), 14 life and medical sciences (genetics, biology, microbiology, psychiatry, clinic, pathology), to earth sciences (geography, geology, remote sensing), social sciences (sociology, psychology, education), 16 and economics (marketing, business). 7,10,23,27 Traditional methods of clustering can be broadly categorized into those of hierarchical, partitioning, density-based, model-based, grid-based, and softcomputing. 17,19,21,25 Inspired by DBSCAN, 6 many density-based clustering methods 5,6,20 have been proposed.…”