Attribute Weighting Based K-Nearest Neighbor Using Gain Ratio

Abstract

Abstract. K-Nearest Neighbor (KNN) is a good classifier, but fro m several studies, the result performance accuracy of KNN still lo wer than other methods. One of the causes of the low accuracy produced, because each attribute has the same effect on the classification process, while some less relevant characteristics lead to mis s-classification of the class assignment for new data. In this research, we proposedAttribute Weighting Based K-Nearest Neighbor Using Gain Rat io as a parameter to see the correlation between each attribute in the data and the Gain Ratio also will be used as the basis for weighting each attribute of the dataset. The accuracy of results is compared to the accuracy acquired fro m the original KNN method using 10-fo ld Cross-Validation with several datasets from the UCI Machine Learning repository and KEELDataset Repository, such as abalone, glass identification,haberman, hayes -roth and water quality status. Based on the result of the test, the proposed method was able to increase the classification accuracy of KNN, where the highest difference of accuracy obtained hayes-roth dataset is worth 12.73%, and the lowest difference of accuracy obtained in the abalone dataset of 0.07%. The average result of the accuracy of all dataset increases the accuracy by 5.33%. (KNN) is one of the effective, simple and performs well method for classification [1][2][3], but from several studies, the result performance accuracy of KNN is lower than other methods. One of them is in the study by [4] which compared performance between support vector machine (SVM) and KNN. The result of their research seen that performance of SVM better than KNN, where the value accuracy obtained by SVM of 82.54% while the value obtained from KNN of 79.22%. Another study by [5] compared K-Nearest Neighbor (KNN) and Artificial Neural Network (ANN). The results of their study seen that the performance of ANN (with 5 hidden layers) better than KNN, where the result of accuracy is 90.5%. Research by [6] compared the performance of Naïve Bayes, Decision Tree and K-Nearest Neighbor (KNN). The results show that Naïve Bayes has the best accuracy in classification compared to Decision Tree Tree and K-Nearest Neighbor (KNN) with an average accuracy of 73.7%. whilethe average accuracy of Decision Tree and KNN respectively 58.9% and 56.7%. In the study by [7] compared KNN and Naïve Bayes in diagnosing heart disease. Results obtained by the value of accuracy Naïve Bayes of 79.62% while the average value of accuracy on KNN is only 64.85% whenk = 10. Introduction K-Nearest Neighbor

A Novel Statistical Theoretical Split Metric for Decision Tree Classification

“…4. Hayes Roth dataset [12] The dataset consists 160 instances, five attributes and three classes on categorization of human subjects.…”

“…For F1score (Table 4) except Gini-index for Hayes-Roth dataset (0.81), the proposed measure outperforms IG and Gini: 0.88 for zoo, 0.97 for Breast, 0.79 for Car evaluation. In Height Table 5, proposed is equal to IG and Gini-index for Hayes-Roth (8) and Car Evaluation (12). However, for zoo (6) and breast dataset (6), IG is better.…”

Implementation Internet of Things for Feeding Catfish Water Quality Analysis Using Linear Regression and K-Nearest Neighbor

“…For each feature, the minimum value of the feature is changed to 0, the maximum value is changed to 1, and every other value is converted to a decimal between 0 and 1. The application of this method can be seen in the model equation 7 as follows [13]:…”

Separability-based Quadratic Feature Transformation to Improve Classification Performance

“…Since this technique is directly geared towards maximizing the fisher score, while the mean and variance of each class can be easily calculated, it has the potential to be applied to multi-class data. Various feature quality test metrics, such as Information Gain [28]- [30], Gain ratio [31], Gini Decrease [32], Anova [33], [34], Chi-Square [35], ReliefF [36], [37], and Fast Correlation-Based Feature selection (FCBF) [38], [39], are used to test the feature quality of the proposed Box-Cox transformation and Quadratic transformation, before finally comparing their respective performance.…”