Cost-efficient implementation of k-NN algorithm on multi-core processors

Ahmadzadeh, Armin; Mirzaei, Reza; Madani, Hatef; Shobeiri, Mohammad; Sadeghi, Mahsa; Gavahi, Mohsen; Jafari, Kianoush; Aznaveh, Mohsen Mahmoudi; Gorgin, Saeid

doi:10.1109/memcod.2014.6961863

Cited by 7 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To alleviate the aforementioned issues, two approaches have been proposed: (a) Reducing the amount of distance measurement computations through algorithmic techniques that use efficient data structures or approximate computing [Chen19]. (b) Implementation techniques that concentrate on inherent high parallelism of k-NN algorithm based on different types of hardware platforms, e.g., multicore processors [Ahma14], graphical processor units (GPU) [Gava15], Field Programmable Gate Array (FPGA) [Lu20], and Application Specific Integrated Circuit (ASIC) [Bout18]. While each platform has pros and cons, FPGA, by providing an acceptable trade-off, offers programmability and flexibility of processors besides the considerable performance and low power consumption of ASICs [Said21].…”

Section: -Introductionmentioning

confidence: 99%

Efficient Hardware Accelerators for k-Nearest Neighbors Classification using Most Significant Digit First Arithmetic

Gorgin,

Gholamrezaei,

Lee

2023

Preprint

Self Cite

View full text Add to dashboard Cite

k-Nearest Neighbors (k-NN) is one of the most widely used classification algorithms in real-world machine learning applications such as computer vision, speech recognition, and data mining. Massive high-dimensional datasets, reasonable accuracy of results, and adequate response time are regarded as the most challenging aspects of the k-NN implementation, which are exacerbated by the exponential increase in dataset size and the feature dimension of each data point. In this paper, we leverage the parallelism and digit-level pipelining opportunities offered by FPGA devices and Online arithmetic to address such issues for k-NN classification based on two different distance metrics: Manhattan and Euclidean. In these designs, all the necessary operations for measuring distances and sorting are performed on serially arriving data at no or minimal hardware cost. Due to serial computation, the size of the classifier instance and its memory footprint are reduced, leading to more parallel instances for our target devices. Furthermore, we dynamically terminate unnecessary computations upon detection to reduce power consumption, which is possible in more than half of cases on average. The proposed k-NN implementations are the first hardware accelerator designs using Online arithmetic on FPGA effectively. Based on implementation results, our proposed k-NN implementation based on Manhattan (Euclidean) distance provides 1.72x to 2.23x (1.06x to 2.76x) Speedup compared to state-of-the-art designs, while these improvements are 270.79x to 739.51x (167.0x to 725.8x) when compared to implementation on a single core of the ARM Cortex-A9 CPU.

show abstract

Section: -Introductionmentioning

confidence: 99%

Efficient Hardware Accelerators for k-Nearest Neighbors Classification using Most Significant Digit First Arithmetic

Gorgin,

Gholamrezaei,

Lee

2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A throughout study on GPU-based parallel versions of several machine learning algorithms is carried out in [8], including implementations for well-known classifiers such as neural networks and support vector machines. In contrast, classifier implementations for multi-core CPU are scattered in the literature; for example the following, just to cite recent ones per each classifier category: A dissimilarity-based classifier -parallel implementation of the k nearest neighbor rule [9] tested on machines having from 2 up to 60 processing cores; A probabilistic classifier -the implementation in [10] of the naïve Bayes classifier, whose authors evaluated their algorithm on the publicly available KDD CUP 99 dataset; A geometric classifierthe so-called scaling version developed in [11] for support vector machines, that was exhaustively studied for several threads/cores ratios. Nonetheless, to the best of our knowledge, neither NFL nor RNFLS have been studied for parallel implementations except for our own preliminary attempts: [12].…”

Section: Introductionmentioning

confidence: 99%

Leave-one-out evaluation of the nearest feature line and the rectified nearest feature line segment classifiers using Multi-core architectures

2018

View full text Add to dashboard Cite

In this paper we present the parallelization of the leave-one-out test: a reproducible test that is, in general, computationally expensive. Parallelization was implemented on multi-core multi-threaded architectures, using the Flynn Single Instruction Multiple Data taxonomy. This technique was used for the preprocessing and processing stages of two classification algorithms that are oriented to enrich the representation in small sample cases: the nearest feature line (NFL) algorithm and the rectified nearest feature line segment (RNFLS) algorithm. Results show an acceleration of up to 18.17 times with the smallest dataset and 29.91 times with the largest one, using the most costly algorithm (RNFLS) whose complexity is O(n 4 ). The paper also shows the pseudo-codes of the serial and parallel Universidad EAFIT 75|LOO Evaluation of the NFL and the RNFLS Classifiers Using Multi-core Architectures algorithms using, in the latter case, a notation that describes the way the parallelization was carried out as a function of the threads.Keywords: Multi-core computing; classification algorithms; leave-one-out test.Evaluación leave-one-out de los clasificadores de la línea de características más cercana y del segmento de línea rectificado más cercano usando arquitecturas multi-núcleo ResumenPresentamos en este artículo la paralelización de la prueba leave-one-out, la cual es una prueba repetible pero que, en general, resulta costosa computacionalmente. La paralelización se implementó sobre arquitecturas multinúcleo con múltiples hilos, usando la taxonomía Flynn Single Instruction Multiple Data. Esta técnica se empleó para las etapas de preproceso y proceso de dos algoritmos de clasificación que están orientados a enriquecer la representación en casos de muestra pequeña: el algoritmo de la línea de características más cercana (NFL) y el algoritmo del segmento de línea rectificado más cercano (RNFLS). Los resultados obtenidos muestran una aceleración de hasta 18.17 veces con el conjunto de datos mas pequeño y de 29.91 veces con el conjunto de datos más grande, empleando el algoritmo más costoso -RNFLS-cuya complejidad es O(n 4 ). El artículo muestra también los pseudocódigos de los algoritmos seriales y paralelos empleando, en este último caso, una notación que describe la manera como se realizó la paralelización en función de los hilos.Palabras clave: Computación con múltiples núcleos; algoritmos de clasificación; prueba leave-one-out.

show abstract

Vidushi: Parallel Implementation of Alpha Miner Algorithm and Performance Analysis on CPU and GPU Architecture

Kundra

Juneja

Sureka

2016

Business Process Management Workshops

View full text Add to dashboard Cite

Cost-efficient implementation of k-NN algorithm on multi-core processors

Cited by 7 publications

References 5 publications

Efficient Hardware Accelerators for k-Nearest Neighbors Classification using Most Significant Digit First Arithmetic

Efficient Hardware Accelerators for k-Nearest Neighbors Classification using Most Significant Digit First Arithmetic

Leave-one-out evaluation of the nearest feature line and the rectified nearest feature line segment classifiers using Multi-core architectures

Vidushi: Parallel Implementation of Alpha Miner Algorithm and Performance Analysis on CPU and GPU Architecture

Contact Info

Product

Resources

About