Mapreduce-Based Distributed Clustering Method Using CF⁺ Tree

Abstract

Clustering exceptionally large data sets is becoming a major challenge in data analytics with the continuous increase in their size. Summary-based clustering methods and distributed computing frameworks such as MapReduce can efficiently handle this challenge. These methods include BIRCH and its extension CF +-ERC. CF +-ERC can reduce the clustering time of large data sets by utilizing the structure of a CF + tree. However, CF +-ERC is a sequential clustering method, so it cannot be used with multiple machines to reduce the clustering time. In this study, we propose a novel MapReduce-based distributed clustering method called CF +-ERC on MapReduce (CF + ERC_MR). It builds a CF + tree for clustering an exceptionally large data set with a given threshold and finds the final clusters using MapReduce, which significantly reduces the clustering time. Further, our method is scalable with respect to the number of machines. The efficacy of this method is validated through not only its theoretical analysis but also indepth experimental analysis of exceptionally large synthetic and real data sets. The experimental results demonstrate that the clustering speed of our approach is far superior to that of the existing clustering methods.

DDCM ： A Decentralized Density Clustering and Its Results Gathering Approach

“…The first one is distributed clustering based on MapReduce [11]- [15]. Its main idea is to adaptively improve classical clustering algorithms using MapReduce, and complete the calculation in parallel.…”

“…As a comparison object, we choose MapReduce-Based Density Clustering (MBDC) [11][30] as a benchmark. To facilitate the experiments, we improve MBDC algorithm as follows.…”