Cost-Free Learning for Support Vector Machines with a Reject Option

Abstract

In this work, we investigate into the abstaining classification of binary support vector machines (SVMs) based on mutual information (MI). We obtain the reject rule by maximizing the MI between the true labels and the predicted labels, which is a post-processing method. The gradient and Hessian matrix of MI are derived explicitly so that Newton method is used for the optimization which converges very fast. Different from the existing reject rules of SVM, the present MIbased reject rule does not require any explicit cost information and is under the framework of cost-free learning. As a matter of fact, the cost information embedded in MI can also be derived from the method, which provides an objective or initial reference to users if they want to apply cost-sensitive learning. Numerical results confirm the benefits of the proposed MIbased reject rule in comparison with other reject rules of SVM.

An asymmetric stagewise least square loss function for imbalanced classification

“…Overally, experimental results confirm the effectiveness of the ASLS loss in imbalanced classification. In the future, we will further study the optimization and sparseness problems of the ASLS loss and compare the ASLS loss to the emerging C-loss function [8] and cost-free leaning [39], [40].…”

Analysis of recent advancements in support vector machine

“…Xu and Hu investigate the classification theory based on mutual information (MI). 22 The authors have proposed a reject rule for the SVM based upon MI optimal. MI-optimal rejection rule is defined as…”

“…The authors have concluded that to avoid severe losses, which can be raised by errors, need to apply to reject option during the classification. 22 Optimization in SVMs is not enough to enhance the performance of classification. 22 Emerging the other optimization techniques like particle swarm optimization (PSO) in SVM may enhance the performance.…”

“…22 Optimization in SVMs is not enough to enhance the performance of classification. 22 Emerging the other optimization techniques like particle swarm optimization (PSO) in SVM may enhance the performance.…”

“…Although, Xu and Hu have proposed the rejection rules to reject the features to make SVM faster but not dependable. 22 Augmentation in dimension is a natural extension to SVM in the present scenario using a kernel. However, Augmentation in dimensionality may raise the numerical complexity.…”