A Proximity Weighted Evidential k Nearest Neighbor Classifier for Imbalanced Data

Abstract

In k Nearest Neighbor (kNN) classifier, a query instance is classified based on the most frequent class of its nearest neighbors among the training instances. In imbalanced datasets, kNN becomes biased towards the majority instances of the training space. To solve this problem, we propose a method called Proximity weighted Evidential kNN classifier. In this method, each neighbor of a query instance is considered as a piece of evidence from which we calculate the probability of class label given feature values to provide more preference to the minority instances. This is then discounted by the proximity of the neighbor to prioritize the closer instances in the local neighborhood. These evidences are then combined using Dempster-Shafer theory of evidence. A rigorous experiment over 30 benchmark imbalanced datasets shows that our method performs better compared to 12 popular methods. In pairwise comparison of these 12 methods with our method, in the best case, our method wins in 29 datasets, and in the worst case it wins in least 19 datasets. More importantly, according to Friedman test the proposed method ranks higher than all other methods in terms of AUC at 5% level of significance.

Use of relevancy and complementary information for discriminatory gene selection from high-dimensional gene expression data

“…Selection of the informative features in the MGS step was implemented using MATLAB and ranking these informative features in MGS f and MGS rf step was implemented using Python with the package scikit-learn [ 41 ]. To evaluate the performance of the proposed and existing methods, different classifiers such as SVM, RF classifiers, XGboost [ 42 ], PE k NN [ 43 ] can be used. In this paper, we only use two simple classifiers namely SVM (linear kernel) and Random Forest to compare different methods.…”

Wind turbine fault detection and isolation robust against data imbalance using KNN

“…Many references confirm that the performance of KNN is affected by data imbalance. 21,24,33 Conceptually the KNN algorithm calculates the (Euclidian) distances of the training data set observations from a validation sample (i.e., to be labeled) and assigns the majority label of KNNs of the validation sample to it. Therefore, if the observations of each class (in the feature space of the data set) have little distance from each other and a small parameter K is selected, the performance of this algorithm will not be affected by data imbalance.…”

“…Although KNN is a simple and accurate algorithm it has some weaknesses such as being biased toward majority observations when facing data imbalance. 21 Several types of research have been performed to improve KNN performance against data imbalance by using oversampling, 22 boosting-byresample strategy, 23 misclassification cost, 24 and so forth. This algorithm has been used for fault detection in a wide area of subjects such as power systems, 25,26 railway point systems, 27 nuclear power plants, 28 and especially WTs, 20,[29][30][31] however, the focus of previous FDI works is not mainly on data imbalance challenge.…”

“…KNN is a nonparametric supervised learning algorithm that can be used for regression and classification predictions. Although KNN is a simple and accurate algorithm it has some weaknesses such as being biased toward majority observations when facing data imbalance 21 . Several types of research have been performed to improve KNN performance against data imbalance by using oversampling, 22 boosting‐by‐resample strategy, 23 misclassification cost, 24 and so forth.…”

Comparison of machine learning techniques to handle imbalanced COVID-19 CBC datasets

“…Although the kNN algorithm is a versatile technique for classification tasks, it has some drawbacks, such as determining a secure way of choosing the k parameter, being sensitive to the similarity (distance) function used (Kotsiantis, Zaharakis & Pintelas, 2006), and a large amount of storage for large datasets (Harrington, 2012). As the kNN considers the most frequent class of its nearest neighbors, it is intuitive to conclude that for imbalanced datasets, the method will bias the results towards the majority class in the training dataset (Kadir et al, 2020).…”