Although the class-imbalance classification problem has caught a huge amount of attention, hyperparameter optimisation has not been studied in detail in this field. Both classification algorithms and resampling techniques involve some hyperparameters that can be tuned. This paper sets up several experiments and draws the conclusion that, compared to using default hyperparameters, applying hyperparameter optimisation for both classification algorithms and resampling approaches can produce the best results for classifying the imbalanced datasets. Moreover, this paper shows that data complexity, especially the overlap between classes, has a big impact on the potential improvement that can be achieved through hyperparameter optimisation. Results of our experiments also indicate that using resampling techniques cannot improve the performance for some complex datasets, which further emphasizes the importance of analyzing data complexity before dealing with imbalanced datasets.

Although the anomaly detection problem can be considered as an extreme case of class imbalance problem, very few studies consider improving class imbalance classification with anomaly detection ideas. Most data-level approaches in the imbalanced learning domain aim to introduce more information to the original dataset by generating synthetic samples. However, in this paper, we gain additional information in another way, by introducing additional attributes. We propose to introduce the outlier score and four types of samples (safe, borderline, rare, outlier) as additional attributes in order to gain more information on the data characteristics and improve the classification performance. According to our experimental results, introducing additional attributes can improve the imbalanced classification performance in most cases (6 out of 7 datasets). Further study shows that this performance improvement is mainly contributed by a more accurate classification in the overlapping region of the two classes (majority and minority classes). The proposed idea of introducing additional attributes is simple to implement and can be combined with resampling techniques and other algorithmiclevel approaches in the imbalanced learning domain.

MethodSeed Classifier Resampler Res.Group GM Grid 9 RF No resampling No resampling 0.9864 Grid 18 RF No resampling No resampling 0.9864 Grid 27 RF No resampling No resampling 0.9864 Grid 29 RF No resampling No resampling 0.9864 Grid 36 RF OneSidedSelection Under resampling 0.9864TABLE S-9 "IRIS0" Method Seed Classifier Resampler Res.Group GM Grid 9 SVM No resampling No resampling 1.0 Grid 18 SVM No resampling No resampling 1.0 Grid 27 SVM No resampling No resampling 1.0 Grid 29 SVM No resampling No resampling 1.0 Grid 36 SVM No resampling No resampling 1.0 Grid 39 SVM No resampling No resampling 1.0 Grid 59 SVM No resampling No resampling 1.0 Grid 79 SVM No resampling No resampling 1.0 Grid 90 SVM No resampling No resampling 1.0 Grid 109 SVM No resampling No resampling 1.0 Random 9 RF ClusterCentroids Under resampling

No abstract

Although over 90 oversampling approaches have been developed in the imbalance learning domain, most of the empirical study and application work are still based on the "classical" resampling techniques. In this paper, several experiments on 19 benchmark datasets are set up to study the efficiency of six powerful oversampling approaches, including both "classical" and new ones. According to our experimental results, oversampling techniques that consider the minority class distribution (new ones) perform better in most cases and RACOG gives the best performance among the six reviewed approaches. We further validate our conclusion on our real-world inspired vehicle datasets and also find applying oversampling techniques can improve the performance by around 10%. In addition, seven data complexity measures are considered for the initial purpose of investigating the relationship between data complexity measures and the choice of resampling techniques. Although no obvious relationship can be abstracted in our experiments, we find F1v value, a measure for evaluating the overlap which most researchers ignore, has a strong negative correlation with the potential AUC value (after resampling).