A CUDA-based parallel implementation of K-nearest neighbor algorithm

Liang, Shenshen; Liu, Ying; Wang, Cheng; Jian, Liheng

doi:10.1109/cyberc.2009.5399145

Cited by 26 publications

(21 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Since KNN is a fundamental algorithm used in a variety of machine learning contexts, a significant amount of time has gone into making it run fast on parallel hardware [5], [6], [7]. However, most of this work has been done to reduce algorithmic complexity or parallelization in a shared memory setting.…”

Section: Introductionmentioning

confidence: 99%

“…Such data structures work well in a variety of scientific applications where the dimensionality of data is not very high, and are critical for large data sets where we cannot afford to scan the entire data set for each query. Furthermore, parallel algorithms that can utilize multiple cores of a single shared memory machine to speed up kd-tree based KNN have been developed [5], [6], [7]. However, there has not been much work on parallelizing this in a distributed setting.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PANDA: Extreme Scale Parallel K-Nearest Neighbor on Distributed Architectures

Patwary¹,

Satish

Sundaram

et al. 2016

2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

View full text Add to dashboard Cite

Abstract-Computing k-Nearest Neighbors (KNN) is one of the core kernels used in many machine learning, data mining and scientific computing applications. Although kd-tree based O(log n) algorithms have been proposed for computing KNN, due to its inherent sequentiality, linear algorithms are being used in practice. This limits the applicability of such methods to millions of data points, with limited scalability for Big Data analytics challenges in the scientific domain. In this paper, we present parallel and highly optimized kdtree based KNN algorithms (both construction and querying) suitable for distributed architectures. Our algorithm includes novel approaches for pruning search space and improving load balancing and partitioning among nodes and threads. Using TB-sized datasets from three science applications: astrophysics, plasma physics, and particle physics, we show that our implementation can construct kd-tree of 189 billion particles in 48 seconds on utilizing ∼50,000 cores. We also demonstrate computation of KNN of 19 billion queries in 12 seconds. We demonstrate almost linear speedup both for shared and distributed memory computers. Our algorithms outperforms earlier implementations by more than order of magnitude; thereby radically improving the applicability of our implementation to state-of-the-art Big Data analytics problems. In addition, we showcase performance and scalability on the recently released Intel R Xeon Phi TM processor showing that our algorithm scales well even on massively parallel architectures.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

PANDA: Extreme Scale Parallel K-Nearest Neighbor on Distributed Architectures

Patwary¹,

Satish

Sundaram

et al. 2016

2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

View full text Add to dashboard Cite

show abstract

“…General parallel algorithms have been devised for sorting [26], clustering [8,27], classification [28,29], and neural networks training [30]. Our work also takes a parallel design and is implemented cooperatively on the CPU and GPU, referencing the implementation of the previous work.…”

Section: Related Workmentioning

confidence: 99%

Big Data Management with Incremental K-Means Trees–GPU-Accelerated Construction and Visualization

Wang

Zelenyuk

Imre³

et al. 2017

Informatics

View full text Add to dashboard Cite

While big data is revolutionizing scientific research, the tasks of data management and analytics are becoming more challenging than ever. One way to remit the difficulty is to obtain the multilevel hierarchy embedded in the data. Knowing the hierarchy enables not only the revelation of the nature of the data, it is also often the first step in big data analytics. However, current algorithms for learning the hierarchy are typically not scalable to large volumes of data with high dimensionality. To tackle this challenge, in this paper, we propose a new scalable approach for constructing the tree structure from data. Our method builds the tree in a bottom-up manner, with adapted incremental k-means. By referencing the distribution of point distances, one can flexibly control the height of the tree and the branching of each node. Dimension reduction is also conducted as a pre-process, to further boost the computing efficiency. The algorithm takes a parallel design and is implemented with CUDA (Compute Unified Device Architecture), so that it can be efficiently applied to big data. We test the algorithm with two real-world datasets, and the results are visualized with extended circular dendrograms and other visualization techniques.

show abstract

“…In addition, increased use of technology has resulted in a wealth of digital trails generated by learners, providing large volumes of trace data collected during the learning process (Knight et al, 2013). Many data mining algorithms have implementations adapted for this big-data environment, for example, Decision Tree (Ben-Haim & TomTov, 2010), k-NN (Liang et al, 2009), Neural Networks (Gu et al, 2013, SVM and regression (Luo et al, 2012), and supporting tools are available (Prekopcsák et al, 2011), facilitating quick analysis and feedback (Siemens & Long, 2011). Recent developments in learning analytics frameworks (e.g., the learning warehouse, Buckingham Shum & Deakin Crick, 2012) illustrate the potential for learning analytics to support automation of the full life cycle from data gathering through to deployment of recommendations and interventions based on analysis results.…”

Section: Benefits Of Greater Collaboration Between Educational Psychomentioning

confidence: 99%

A Review of Psychometric Data Analysis and Applications in Modelling of Academic Achievement in Tertiary Education

Gray¹

2014

Learning Analytics

View full text Add to dashboard Cite

Increasing college participation rates, and diversity in student population, is posing a challenge to colleges in their attempts to facilitate learners achieve their full academic potential. Learning analytics is an evolving discipline with capability for educational data analysis that could enable better understanding of learning process, and therefore mitigate these challenges. The outcome from such data analysis will be dependent on the range, type, and quality of available data and the type of analysis performed. This study reviewed factors that could be used to predict academic performance, but which are currently not systematically measured in tertiary education. It focused on psychometric factors of ability, personality, motivation, and learning strategies. Their respective relationships with academic performance are enumerated and discussed. A case is made for their increased use in learning analytics to enhance the performance of existing student models. It is noted that lack of independence, linear additivity, and constant variance in the relationships between psychometric factors and academic performance suggests increasing relevance of data mining techniques, which could be used to provide useful insights on the role of such factors in the modelling of learning process.

show abstract

A CUDA-based parallel implementation of K-nearest neighbor algorithm

Cited by 26 publications

References 8 publications

PANDA: Extreme Scale Parallel K-Nearest Neighbor on Distributed Architectures

PANDA: Extreme Scale Parallel K-Nearest Neighbor on Distributed Architectures

Big Data Management with Incremental K-Means Trees–GPU-Accelerated Construction and Visualization

A Review of Psychometric Data Analysis and Applications in Modelling of Academic Achievement in Tertiary Education

Contact Info

Product

Resources

About