A Novel Ensemble Framework Based on K-Means and Resampling for Imbalanced Data

Abstract

Imbalanced classification is one of the most important problems of machine learning and data mining, existing in many real datasets. In the past, many basic classifiers such as SVM, KNN, and so on have been used for imbalanced datasets in which the number of one sample is larger than that of another, but the classification effect is not ideal. Some data preprocessing methods have been proposed to reduce the imbalance ratio of data sets and combine with the basic classifiers to get better performance. In order to improve the whole classification accuracy, we propose a novel classifier ensemble framework based on K-means and resampling technique (EKR). First, we divide the data samples in the majority class into several sub-clusters using K-means, k-value is determined by Average Silhouette Coefficient, and then adjust the number of data samples of each sub-cluster to be the same as that of the minority classes through resampling technology, after that each adjusted sub-cluster and the minority class are combined into several balanced subsets, the base classifier is trained on each balanced subset separately, and finally integrated into a strong ensemble classifier. In this paper, the extensive experimental results on 16 imbalanced datasets demonstrate the effectiveness and feasibility of the proposed algorithm in terms of multiple evaluation criteria, and EKR can achieve better performance when compared with several classical imbalanced classification algorithms using different data preprocessing methods.

Experimental Study and Comparison of Imbalance Ensemble Classifiers with Dynamic Selection Strategy

“…The challenges of imbalanced classification and its prevalence have resulted in much research in this area. An effective solution is to design classifiers based on ensemble approaches [ 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ]. Ensemble schemes for imbalanced classification have been developed from the perspectives of data decomposition, cost-sensitive schemes, sampling methods, and dynamic selection.…”

Special Issue on Data Preprocessing in Pattern Recognition: Recent Progress, Trends and Applications

“…A total of ten high-quality and peer-reviewed papers form this Special Issue, covering the following topics: class imbalance [1][2][3][4][5][6], big data preprocessing [1], prototype selection [7,8], variable selection [9] and clustering data on arbitrary shape [10].…”

“…Rendon et al [1] propose to mitigate the unbalance of multi-class big datasets using a hybrid method, conformed by a well-known oversampling technique and a prototype selection method, applied in the artificial neural network's output domain as well as the feature space. Duan et al [2] propose a two-step solution for two-class problems using a novel classifier ensemble framework based on K-means and the oversampling technique called ADASYIN. Rangel-Díaz-dela-Vega et al [3] performed an experimental study on the behavior of four associative classifiers trained on resampled imbalanced credit scoring datasets.…”