Efficient Map/Reduce-Based DBSCAN Algorithm with Optimized Data Partition

Dai, Bi-Ru; Lin, I-Chang

doi:10.1109/cloud.2012.42

Cited by 75 publications

(48 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This overlap is needed because of the possibility that the data points of a cluster are spread across different partitions, so there should be an overlap (a joint or boundary region) between adjacent partitions. Selecting a 2ε-wide boundary region ensures sufficient information for the merge phase [9]. As an example, consider a twodimensional space in which each of the dimensions is normalized between 0 and 1, and the interval of each dimension is divided into two portions.…”

Section: Create a Grid For The Data Spacementioning

confidence: 99%

“…In addition, a core point reduction approach is proposed and implemented for more efficient and yet accurate data summarization and regeneration. Note that MapReduce-based implementation of DBSCAN algorithm has been studied in [8,9,10] that are relying on different partitioning strategies. In this thesis, a grid-based partitioning approach is used for an implementation of DBSCAN algorithm that is compatible with GMM summarization, which are both completely developed in the MapReduce framework.…”

Section: Contributionsmentioning

confidence: 99%

“…There should be a joint area between adjacent partitions, allowing retrieval of any cluster which is spread across adjacent partitions. The points in the joint areas between adjacent partitions should be copied to all adjacent partitions; these are referred to as boundary (border) points [9]. There are different partitioning strategies available [8,9,10], which are explained following.…”

Section: Partitioningmentioning

confidence: 99%

“…This reduces the execution time. The PRBP algorithm in [9] is not implemented with the MapReduce framework, so the execution time can be high for large amounts of data. As well, the steps to find merge lists and relabel data points are not implemented with the MapReduce framework.…”

Section: Density-based Clustering Approaches Implemented In the Maprementioning

confidence: 99%

“…The algorithm in [9] performs the distributed DBSCAN in four steps. First, the dataset is partitioned for distribution among nodes.…”

Section: Density-based Clustering Approaches Implemented In the Maprementioning

confidence: 99%

See 4 more Smart Citations

Distributed Gaussian Mixture Model Summarization Using the MapReduce Framework

Esmaeilpour

Bigdeli

Cheraghchi

et al. 2016

Advances in Artificial Intelligence

View full text Add to dashboard Cite

With an accelerating rate of data generation, sophisticated techniques are essential to meet scalability requirements. One of the promising avenues for handling large datasets is distributed storage and processing. Hadoop is a well-known framework for distributed storage and processing. Further, data summarization is a useful concept for managing large datasets. Data summarization techniques are intended to produce compact yet representative summaries for the entire dataset.Consolidation of these tools can allow a distributed implementation of data summarization. In this thesis, this goal is achieved by proposing and implementing a distributed Gaussian Mixture Model Summarization using the MapReduce framework (MR-SGMM). The main purpose of the proposed method is to summarize a dataset with a density-based clustering algorithm called DBSCAN algorithm, and then summarize each discovered cluster using the SGMM approach in a distributed manner. Testing the implementation with synthetic and real datasets is used to demonstrate its validity and efficiency.ii

show abstract

Section: Create a Grid For The Data Spacementioning

confidence: 99%

Section: Contributionsmentioning

confidence: 99%