Imbalanced Data Sample Pruning Algorithm Based on Dynamic threshold K Nearest Neighbor

Abstract

Abstract. This paper present a new sample pruning algorithm based on dynamic threshold KNN to deal with the complexity and overlapping problem of imbalanced data set. The phenomenon of data complexity and overlapping will reduce the classification performance and generalization ability of SVM classifier. Especially in imbalanced data set, this phenomenon is more obvious due to the quantity difference between positive and negative samples. We apply KNN to prune the training samples according to the similarities of each sample between its K labeled nearest neighbors and select different dynamic threshold to adapt the characteristic of imbalanced data set. The comparative experiments show that our algorithm can effectively improve the SVM classification performance in imbalanced data set.