A Dataset-Driven Parameter Tuning Approach for Enhanced K-Nearest Neighbour Algorithm Performance

Abstract

The number of Neighbours (k) and distance measure (DM) are widely modified for improved kNN performance. This work investigates the joint effect of these parameters in conjunction with dataset characteristics (DC) on kNN performance. Euclidean; Chebychev; Manhattan; Minkowski; and Filtered distances, eleven k values, and four DC, were systematically selected for the parameter tuning experiments. Each experiment had 20 iterations, 10-fold cross-validation method and thirty-three randomly selected datasets from the UCI repository. From the results, the average root mean squared error of kNN is significantly affected by the type of task (p9000, as optimal performance pattern for classification tasks. For regression problems, the experimental configuration should be7000≤SS≤9000; 4≤number of attributes ≤6, and DM = 'Filtered'. The type of task performed is the most influential kNN performance determinant, followed by DM. The variation in kNN accuracy resulting from changes in k values only occurs by chance, as it does not depict any consistent pattern, while its joint effect of k value with other parameters yielded a statistically insignificant change in mean accuracy (p>0.5). As further work, the discovered patterns would serve as the standard reference for comparative analytics of kNN performance with other classification and regression algorithms.

Predicting the risk of primary Sjögren's syndrome with key N7-methylguanosine-related genes: A novel XGBoost model

“…SVM, for instance, is renowned for its ability to handle high-dimensional data and its robustness to outliers, but it may suffer from sensitivity to kernel function selection and parameter tuning [ 29 ]. KNN, on the other hand, is a simple yet effective method for classification and regression, but its performance can be heavily influenced by the choice of the number of neighbors and the distance metric [ 30 ]. RF, as an ensemble method, often achieves good performance by combining the predictions of multiple decision trees.…”

An Artificial Neural Networks (ANN) Approach for 3 Degrees of Freedom Motion Controlling

“…As in the forward propagation Equation 9 and backpropagation formulae, the weights and biases are computed and modified at the k + 1st step (Equation 10). The error produced by the network, based on the percentage absolute error (APE) and the average percentage absolute error, is the condition for ending learning for the training process (MAPE) [27]. Following training, we move on to evaluate the network's error in order to select the most effective network for forecasting.…”

Advanced Anomaly Detection in Manufacturing Processes: Leveraging Feature Value Analysis for Normalizing Anomalous Data

“…To predict product defects using data collected from the factory that had undergone preprocessing, a total of five machine learning-based classification models were utilized: K-Nearest Neighbors Classifier, Decision Trees Classifier, Random Forest Classifier, Extra Trees Classifier, and Gradient Boosting Classifier. In employing each model, parameters such as the number of neighbors (n_neighbors), the number of trees (n_estimators), the maximum depth of the trees (max_depth), and the learning rate (learning_rate) were meticulously controlled to ensure a fair comparison of accuracy across models [32][33][34][35][36]. Furthermore, the Grid Search method was employed to identify the most optimal combination of parameters for each model [37].…”