Building a Biased Least Squares Support Vector Machine Classifier for Positive and Unlabeled Learning

Abstract

Learning from positive and unlabeled examples (PU learning) is a special case of semi-supervised binary classification. The key feature of PU learning is that there is no labeled negative training data, which makes the traditional classification techniques inapplicable. Similar to the idea of Biased-SVM which is one of the most famous classifier, a biased least squares support vector machine classifier (Biased-LSSVM) is proposed for PU learning in this paper. More specifically, we take unlabeled examples as negative examples with noise and build a least squares support vector machine classifier using two penalty parameters p C and n C to weight misclassification errors of positive and negative examples respectively. As we pay more attention to classify as many as positive examples correctly in PU learning, the relationship of parameters p C and n C is p n C C ≥. Compared with Biased-SVM, the proposed classifier has three advantages. First, Biased-LSSVM can reflect the class labels of all examples more sufficiently and accurately than Biased-SVM. Second, Biased-LSSVM is more stable than Biased-SVM because the performance of Biased-LSSVM changes less than that of Biased-SVM over a wide ratio of positive examples in unlabeled examples. Finally, the time complexity of Biased-LSSVM is lower than that of Biased-SVM, where Biased-LSSVM only need to solve liner equations and Biased-SVM is a quadratic programming. The Experiments on two real applications, text classification and bioinformatics classification verify the above opinions and show that Biased-LSSVM is more effective than Biased-SVM and other popular methods, such as EB-SVM, ROC-SVM and S-EM.