Weighted Support Vector Machines with the SCAD Penalty

Abstract

Classification is an important research area as data can be easily obtained even if the number of predictors becomes huge. The support vector machine(SVM) is widely used to classify a subject into a predetermined group because it gives sound theoretical background and better performance than other methods in many applications. The SVM can be viewed as a penalized method with the hinge loss function and penalty functions. Instead of L 2 penalty function Fan and Li (2001) proposed the smoothly clipped absolute deviation(SCAD) satisfying good statistical properties. Despite the ability of SVMs, they have drawbacks of non-robustness when there are outliers in the data. We develop a robust SVM method using a weight function with the SCAD penalty function based on the local quadratic approximation. We compare the performance of the proposed SVM with the SVM using the L 1 and L 2 penalty functions.

Support Vector Machines for Unbalanced Multicategory Classification

“…where V is the weight for the th observation with the th class. A weighted SVM is proposed for the robustness of the SVM which is not sensitive to outlier or leverage points (see [13]). We consider the weight for each class as…”

“…The within group errors can be calculated as the misclassification rate for the th class. Weight (13) gives much more weights on the minority class and the well-classified group got the less weight. The larger values of | (x )| in (13) represent well-classified observations.…”

Selecting Biomarkers for Building Optimal Treatment Selection Rules by Using Kernel Machines

“…Some work has been done to extend the smoothly clipped absolute deviation (SCAD) penalty with weighted linear SVMs with special forms of such weights (see Jung (2013)), but, beyond that, there has not been any targeted investigation of such to our knowledge. These two reasons make these explorations vitally important.…”

“…However, it is worth noting that our setting differs from a simple classification format in two vital aspects: (a) although the treatment selection objective can be rewritten into a (weighted) classification problem (as shown in Section 2), it is still in essence a fundamentally different problem from classification, and feature selection techniques in SVMs have not been studied under this context, and (b) weighted SVM is a more complicated optimization problem than the standard SVM, where the constraint on each support vector varies according to the weight associated with it, and research into feature extraction under this setting has also been fairly limited till now. Some work has been done to extend the SCAD penalty with the weighted linear support vector machines with special forms of such weights (see Jung, 2013), but beyond that, there hasn't been any targeted investigation of such as per our knowledge. These two reasons make these explorations vitally important.…”

Penalized rank regression estimator with the smoothly clipped absolute deviation function

“…One alternative is the least absolute deviation (LAD) estimate. Jung (2011Jung ( , 2013 proposed robust estimators and outlier detection methods in regression models and support vector machine. There are several robust versions of LASSO.…”