No abstract

The correntropy-induced loss (C-loss) function has the nice property of being robust to outliers. In this paper, we study the C-loss kernel classifier with the Tikhonov regularization term, which is used to avoid overfitting. After using the half-quadratic optimization algorithm, which converges much faster than the gradient optimization algorithm, we find out that the resulting C-loss kernel classifier is equivalent to an iterative weighted least square support vector machine (LS-SVM). This relationship helps explain the robustness of iterative weighted LS-SVM from the correntropy and density estimation perspectives. On the large-scale data sets which have low-rank Gram matrices, we suggest to use incomplete Cholesky decomposition to speed up the training process. Moreover, we use the representer theorem to improve the sparseness of the resulting C-loss kernel classifier. Experimental results confirm that our methods are more robust to outliers than the existing common classifiers.

In this work, we investigate into the abstaining classification of binary support vector machines (SVMs) based on mutual information (MI). We obtain the reject rule by maximizing the MI between the true labels and the predicted labels, which is a post-processing method. The gradient and Hessian matrix of MI are derived explicitly so that Newton method is used for the optimization which converges very fast. Different from the existing reject rules of SVM, the present MIbased reject rule does not require any explicit cost information and is under the framework of cost-free learning. As a matter of fact, the cost information embedded in MI can also be derived from the method, which provides an objective or initial reference to users if they want to apply cost-sensitive learning. Numerical results confirm the benefits of the proposed MIbased reject rule in comparison with other reject rules of SVM.