A Novel Ensemble Method for Imbalanced Data Learning: Bagging of Extrapolation-SMOTE SVM

Abstract

Class imbalance ubiquitously exists in real life, which has attracted much interest from various domains. Direct learning from imbalanced dataset may pose unsatisfying results overfocusing on the accuracy of identification and deriving a suboptimal model. Various methodologies have been developed in tackling this problem including sampling, cost-sensitive, and other hybrid ones. However, the samples near the decision boundary which contain more discriminative information should be valued and the skew of the boundary would be corrected by constructing synthetic samples. Inspired by the truth and sense of geometry, we designed a new synthetic minority oversampling technique to incorporate the borderline information. What is more, ensemble model always tends to capture more complicated and robust decision boundary in practice. Taking these factors into considerations, a novel ensemble method, called Bagging of Extrapolation Borderline-SMOTE SVM (BEBS), has been proposed in dealing with imbalanced data learning (IDL) problems. Experiments on open access datasets showed significant superior performance using our model and a persuasive and intuitive explanation behind the method was illustrated. As far as we know, this is the first model combining ensemble of SVMs with borderline information for solving such condition.

Examining the Ability of Artificial Neural Networks Machine Learning Models to Accurately Predict Complications Following Posterior Lumbar Spine Fusion

“…Low occurrence rates in relatively small datasets lead to large class-imbalances that are a significant challenge in medical machine learning. 22,23 To this end, we have trained several supervised machine learning classifiers to predict the probability of postoperative complications in a relatively small dataset (<15,000 patients) that can accurately learn complications with relatively low occurrence rates (<1%). We have rigorously developed and tested our models by employing the best practices in machine learning in this study by performing automated feature selection, L2 regularization, testing on blinded hold-out data sets, and comparing to a standard risk-scoring system to ensure a high standard that is necessary for implementation of machine learning in clinical settings.…”

Designing a Streaming Algorithm for Outlier Detection in Data Mining—An Incrementa Approach

“…A thing to take note when using supervised method for training is imbalanced data: The predictive models developed using conventional machine learning algorithms could be biased and inaccurate because the number of observations in one class of the dataset is significantly lower than the other. To handle imbalanced data, several methods can be used, including resampling, boosting, bagging [17][18][19][20].…”

Using Imbalanced Triangle Synthetic Data for Machine Learning Anomaly Detection

“…ADASYN [He, Bai, Garcia et al (2008)] is an important improvement of SMOTE, which generates the synthetic examples by the proportion of the majority ratio. SVM-SMOTE [Nguyen, Cooper and Kamei (2011);Wang, Luo, Huang et al (2017)] generates artificial support vectors by SMOTE and gets good experimental results. Although these algorithms have different generating tricks, the core generating method is still the selected line segment way.…”