Support vector machine regularization

Abstract

Finding the best decision boundary for a classification problem involves covariance structures, distance measures, and eigenvectors. This article considers how eigenstructures are an inherent part of the support vector machine (SVM) functional basis that encodes the geometric features of a separating hyperplane. SVM learning capacity involves an eigenvector set that spans the parameter space being learned. The linear SVM has been shown to have insufficient learning capacity when the number of training examples exceeds the dimension of the feature space. For this case, an incomplete eigenvector set spans the observation space. SVM architectures based on insufficient eigenstructures lack sufficient learning capacity for good separating hyperplanes. However, proper regularization ensures that two essential types of 'biases' are encoded within SVM functional mappings: an appropriate set of algebraic (and thus geometric) relationships and a sufficient eigenstructure set.

Exploiting Linear Support Vector Machine for Correlation-Based High Dimensional Data Classification in Wireless Sensor Networks

“…Support vectors are defined as the data points that the margin pushes up against or points that are close to the opposing class. Therefore SVM algorithm implies that only these support vectors are essential, whereas other training examples are ignorable [ 18 ] as seen in Figure 2 .…”

“…Advantages of using SVMs include [ 43 ]: They are effective in high dimension spaces [ 18 ]; SVMs use the subset of training points in the decision function on support vector; it is memory efficient; different kernel functions can be specified for various decision functions; and kernel functions can be added together to achieve even more complex hyperplane. SVMs also have some drawbacks such as, if the number of features is greater than the number of samples, the algorithm is likely to give poor performance [ 44 ].…”

Design of Data-Driven Mathematical Laws for Optimal Statistical Classification Systems

“…I have conducted simulation studies which demonstrate that properly regularized, linear kernel SVMs learn optimal linear decision boundaries for any two classes of Gaussian data, where Σ 1 = Σ 2 = Σ (see Reeves, 2009;Reeves and Jacyna, 2011), including completely overlapping data distributions (see Reeves, 2015). I have also conducted simulation studies which demonstrate that properly regularized, second-order, polynomial kernel SVMs learn optimal decision boundaries for data drawn from any two Gaussian distributions, including completely overlapping data distributions (Reeves, 2015).…”

“…Fitting learning machine architectures to unknown functions of data involves multiple and interrelated difficulties. Learning machine architectures are sensitive to algebraic and topological structures that include functionals, reproducing kernels, kernel parameters, and constraint sets (see, e.g., Geman et al, 1992;Burges, 1998;Gershenfeld, 1999;Byun and Lee, 2002;Haykin, 2009;Reeves, 2015) as well as regularization parameters that determine eigenspectra of data matrices (see, e.g., Haykin, 2009;Reeves, 2009;Reeves and Jacyna, 2011;Reeves, 2015). Identifying the correct form of an equation for a statistical model is also a large concern (Daniel and Wood, 1979;Breiman, 1991;Geman et al, 1992;Gershenfeld, 1999;Duda et al, 2001).…”

A Precise Drunk Driving Detection Using Weighted Kernel Based on Electrocardiogram

“…The polynomial kernel achieves good generalization but its learning capacity is relatively low [ 12 ]. Low learning capacity is usually due to the fact that the training data exceeds the dimensionality of feature space and subsequently results in a poor separating hyperplane [ 20 ]. The quadratic kernel and third order polynomial kernel both belong to the polynomial kernel but they are different in the degrees of fitting (Quadratic is 2 and third order polynomial is 3).…”