Density and Distance Based KNN Approach to Classification

Abstract

KNN algorithm is a simple and efficient algorithm developed to solve classification problems. However, it encounters problems when classifying datasets with non-uniform density distributions. The existing KNN voting mechanism may lose essential information by considering majority only and get degraded performance when a dataset has uneven distribution. The other drawback comes from the way that KNN treat all the participating candidates equally when judging upon one test datum. To overcome the weaknesses of KNN, a Region of Influence Based KNN (RI-KNN) is proposed. RI-KNN computes for each training datum region of influence information based on their nearby data (i.e. locality information) so that each training datum can encode some locality information from its region. Information coming from both training and testing stages will contribute to the formation of weighting formula. By solving these two problems, RI-KNN is shown to outperform KNN upon several artificial datasets and real datasets without sacrificing time cost much in nearly all tested datasets.