Multiple Kernel Learning With Minority Oversampling for Classifying Imbalanced Data

Wang, Ling; Wang, Hongqiao; Fu, Guohong

doi:10.1109/access.2020.3046604

“…Other ways of oversampling include, but are not limited to, the work of [91,92,93,94,78,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119] The validation process is what all oversampling methods have in common, which is basically the evaluation of the classifier's performance employed to classify the oversampled datasets using one or more accuracy measures such as Accuracy, Precision, Recall, F-measure, G-mean, Specificity, Kappa, Matthews correlation coefficient (MCC), Area under the ROC Curve (AUC), True positive rate, False negative (FN), False positive (FP), True positive (TP), True negative (TN), and ROC curve. Table 1 lists 72 oversampling methods, including their known names, references, the number of datasets utilized, the number of classes in these datasets, the classifiers employed, and the performance metrics used to validate the classification results after oversampling.…”

Section: Literature Review Of Oversampling Methodsmentioning

confidence: 99%

Stop Oversampling for Class Imbalance Learning: A Critical Review

Hassanat

¹

,

Tarawneh

²

,

Altarawneh

³

2022

Preprint

5

0

View full text Add to dashboard Cite

For the last two decades, oversampling has been employed to overcome the challenge of learning from imbalanced datasets. Many approaches to solving this challenge have been offered in the literature. Oversampling, on the other hand, is a concern. That is, models trained on fictitious data may fail spectacularly when put to real-world problems. The fundamental difficulty with oversampling approaches is that, given a real-life population, the synthesized samples may not truly belong to the minority class. As a result, training a classifier on these samples while pretending they represent minority may result in incorrect predictions when the model is used in the real world. We analyzed a large number of oversampling methods in this paper and devised a new oversampling evaluation system based on hiding a number of majority examples and comparing them to those generated by the oversampling process. Based on our evaluation system, we ranked all these methods based on their incorrectly generated examples for comparison. Our experiments using more than 70 oversampling methods and three imbalanced real-world datasets reveal that all oversampling methods studied generate minority samples that are most likely to be majority. Given data and methods in hand, we argue that oversampling in its current forms and methodologies is unreliable for learning from class imbalanced data and should be avoided in real-world applications.

show abstract

“…Other ways of oversampling include, but are not limited to, the work of [91,92,93,94,78,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119] The validation process is what all oversampling methods have in common, which is basically the evaluation of the classifier's performance employed to classify the oversampled datasets using one or more accuracy measures such as Accuracy, Precision, Recall, F-measure, G-mean, Specificity, Kappa, Matthews correlation coefficient (MCC), Area under the ROC Curve (AUC), True positive rate, False negative (FN), False positive (FP), True positive (TP), True negative (TN), and ROC curve. Table 1 lists 72 oversampling methods, including their known names, references, the number of datasets utilized, the number of classes in these datasets, the classifiers employed, and the performance metrics used to validate the classification results after oversampling.…”

Section: Literature Review Of Oversampling Methodsmentioning

confidence: 99%

Stop Oversampling for Class Imbalance Learning: A Critical Review

Hassanat¹,

Tarawneh²,

Altarawneh³

2022

Preprint

0

View full text Add to dashboard Cite

For the last two decades, oversampling has been employed to overcome the challenge of learning from imbalanced datasets. Many approaches to solving this challenge have been offered in the literature. Oversampling, on the other hand, is a concern. That is, models trained on fictitious data may fail spectacularly when put to real-world problems. The fundamental difficulty with oversampling approaches is that, given a real-life population, the synthesized samples may not truly belong to the minority class. As a result, training a classifier on these samples while pretending they represent minority may result in incorrect predictions when the model is used in the real world. We analyzed a large number of oversampling methods in this paper and devised a new oversampling evaluation system based on hiding a number of majority examples and comparing them to those generated by the oversampling process. Based on our evaluation system, we ranked all these methods based on their incorrectly generated examples for comparison. Our experiments using more than 70 oversampling methods and three imbalanced real-world datasets reveal that all oversampling methods studied generate minority samples that are most likely to be majority. Given data and methods in hand, we argue that oversampling in its current forms and methodologies is unreliable for learning from class imbalanced data and should be avoided in real-world applications.

show abstract

“…When there are irregularities in the imbalanced data (such as small disjuncts, overlapping, and noise [30]) and the data scale is large, applying a single kernel may make the model biased, skew, or misleading. Inspired by the MKL algorithm [31], we construct a low rank approximate multiple kernel framework as follows:…”

Section: Proposed Algorithmsmentioning

confidence: 99%

“…We divide the imbalance dataset D ={( x i , y i )} i =1 n into the minority class set D + ={( x i , +1)} i =1 n + and the majority class set D − ={( x i , −1)} i =1 n − . When there are irregularities in the imbalanced data (such as small disjuncts, overlapping, and noise [ 30 ]) and the data scale is large, applying a single kernel may make the model biased, skew, or misleading. Inspired by the MKL algorithm [ 31 ], we construct a low rank approximate multiple kernel framework as follows:

where

corresponds to the rank- k approximation of each base kernel matrix K m , and d m is the corresponding mixture weight.…”

Section: Proposed Algorithmsmentioning

confidence: 99%

Multi‐Nyström Method Based on Multiple Kernel Learning for Large Scale Imbalanced Classification

Wang

¹

,

Fu

²

2021

Computational Intelligence and Neuroscience

Self Cite

1

0

View full text Add to dashboard Cite

Extensions of kernel methods for the class imbalance problems have been extensively studied. Although they work well in coping with nonlinear problems, the high computation and memory costs severely limit their application to real-world imbalanced tasks. The Nyström method is an effective technique to scale kernel methods. However, the standard Nyström method needs to sample a sufficiently large number of landmark points to ensure an accurate approximation, which seriously affects its efficiency. In this study, we propose a multi-Nyström method based on mixtures of Nyström approximations to avoid the explosion of subkernel matrix, whereas the optimization to mixture weights is embedded into the model training process by multiple kernel learning (MKL) algorithms to yield more accurate low-rank approximation. Moreover, we select subsets of landmark points according to the imbalance distribution to reduce the model’s sensitivity to skewness. We also provide a kernel stability analysis of our method and show that the model solution error is bounded by weighted approximate errors, which can help us improve the learning process. Extensive experiments on several large scale datasets show that our method can achieve a higher classification accuracy and a dramatical speedup of MKL algorithms.

show abstract

Multiple Kernel Learning With Minority Oversampling for Classifying Imbalanced Data

Cited by 8 publications

References 33 publications

Stop Oversampling for Class Imbalance Learning: A Critical Review

Stop Oversampling for Class Imbalance Learning: A Critical Review

Stop Oversampling for Class Imbalance Learning: A Critical Review

Multi‐Nyström Method Based on Multiple Kernel Learning for Large Scale Imbalanced Classification

Contact Info

Product

Resources

About