A Parallel DBSCAN Algorithm Based on Spark

Luo, Guangchun; Luo, Xiaoyu; Gooch, Thomas Fairley; Tian, Ling; Qin, Ke

doi:10.1109/bdcloud-socialcom-sustaincom.2016.85

Cited by 39 publications

(12 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To further verify the effectiveness of MOWOATS in analyzing huge microarray datasets, it was compared to clustering methods that were based on MapReduce and Spark frameworks, such as a MapReduce based K -means method ( MRK ) proposed by Shahrivari & Jalili (2016) and MapReduce based Bee colony clustering method ( MRB ) proposed by Banharnsakun (2017) . Articles that represent clustering methods based on Spark were: K -M algorithm presented in Spark Machine learning Library ( MLK ) proposed by Gopalani & Arora (2015) , Spark DBscan algorithm proposed ( SDB ) by Luo et al (2016) , and DDHFC proposed by Hosseini & Kiani (2019) . All these methods combined SI methods with Big Data frameworks.…”

Section: Resultsmentioning

confidence: 99%

A hybrid multi-objective whale optimization algorithm for analyzing microarray data based on Apache Spark

AbdelAziz

Soliman

Ghany

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

A microarray is a revolutionary tool that generates vast volumes of data that describe the expression profiles of genes under investigation that can be qualified as Big Data. Hadoop and Spark are efficient frameworks, developed to store and analyze Big Data. Analyzing microarray data helps researchers to identify correlated genes. Clustering has been successfully applied to analyze microarray data by grouping genes with similar expression profiles into clusters. The complex nature of microarray data obligated clustering methods to employ multiple evaluation functions to ensure obtaining solutions with high quality. This transformed the clustering problem into a Multi-Objective Problem (MOP). A new and efficient hybrid Multi-Objective Whale Optimization Algorithm with Tabu Search (MOWOATS) was proposed to solve MOPs. In this article, MOWOATS is proposed to analyze massive microarray datasets. Three evaluation functions have been developed to ensure an effective assessment of solutions. MOWOATS has been adapted to run in parallel using Spark over Hadoop computing clusters. The quality of the generated solutions was evaluated based on different indices, such as Silhouette and Davies–Bouldin indices. The obtained clusters were very similar to the original classes. Regarding the scalability, the running time was inversely proportional to the number of computing nodes.

show abstract

Section: Resultsmentioning

confidence: 99%

A hybrid multi-objective whale optimization algorithm for analyzing microarray data based on Apache Spark

AbdelAziz

Soliman

Ghany

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…In Luo et al (2016), a parallel implementation of DBSCAN algorithm (S_ DBSCAN) based on spark is proposed. The algorithm is divided into three stages; partitioning the input data based on random sampling; perform local DBSCAN in parallel to generate partial clusters; merge the partial clusters based on the centroid.…”

Section: Scalable Methodsmentioning

confidence: 99%

Big data clustering techniques based on Spark: a literature review

Saeed

Aghbari

Alsharidah

2020

PeerJ Computer Science

View full text Add to dashboard Cite

A popular unsupervised learning method, known as clustering, is extensively used in data mining, machine learning and pattern recognition. The procedure involves grouping of single and distinct points in a group in such a way that they are either similar to each other or dissimilar to points of other clusters. Traditional clustering methods are greatly challenged by the recent massive growth of data. Therefore, several research works proposed novel designs for clustering methods that leverage the benefits of Big Data platforms, such as Apache Spark, which is designed for fast and distributed massive data processing. However, Spark-based clustering research is still in its early days. In this systematic survey, we investigate the existing Spark-based clustering methods in terms of their support to the characteristics Big Data. Moreover, we propose a new taxonomy for the Spark-based clustering methods. To the best of our knowledge, no survey has been conducted on Spark-based clustering of Big Data. Therefore, this survey aims to present a comprehensive summary of the previous studies in the field of Big Data clustering using Apache Spark during the span of 2010–2020. This survey also highlights the new research directions in the field of clustering massive data.

show abstract

“…In [73], a parallel implementation of DBSCAN algorithm (S_ DBSCAN) based on spark is proposed. The algorithm is divided into three stages; partitioning the input data based on random sampling; perform local DBSCAN in parallel to generate partial clusters; merge the partial clusters based on the centroid.…”

Section: C4 Scalable Methodsmentioning

confidence: 99%

Peer Review #3 of "Big data clustering techniques based on Spark: a literature review (v0.1)"

2020

View full text Add to dashboard Cite

A Parallel DBSCAN Algorithm Based on Spark

Cited by 39 publications

References 7 publications

A hybrid multi-objective whale optimization algorithm for analyzing microarray data based on Apache Spark

A hybrid multi-objective whale optimization algorithm for analyzing microarray data based on Apache Spark

Big data clustering techniques based on Spark: a literature review

Peer Review #3 of "Big data clustering techniques based on Spark: a literature review (v0.1)"

Contact Info

Product

Resources

About