A fast iterative single data approach to training unconstrained least squares support vector machines

Li, Bing; Song, Shiji; Li, Kang

doi:10.1016/j.neucom.2012.11.030

Cited by 9 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model selection is to seek proper values of hyper-parameters commonly by means of cross-validation and grid search [32]. The k-fold cross-validation [12,13] partitions the training data into k disjoint subsets of approximately equal size.…”

Section: Kernel Function and Model Selectionmentioning

confidence: 99%

Product Design Time Forecasting by Kernel-Based Regression with Gaussian Distribution Weights

Shang

Yan

2016

Entropy

View full text Add to dashboard Cite

There exist problems of small samples and heteroscedastic noise in design time forecasts. To solve them, a kernel-based regression with Gaussian distribution weights (GDW-KR) is proposed here. GDW-KR maintains a Gaussian distribution over weight vectors for the regression. It is applied to seek the least informative distribution from those that keep the target value within the confidence interval of the forecast value. GDW-KR inherits the benefits of Gaussian margin machines. By assuming a Gaussian distribution over weight vectors, it could simultaneously offer a point forecast and its confidence interval, thus providing more information about product design time. Our experiments with real examples verify the effectiveness and flexibility of GDW-KR.

show abstract

Section: Kernel Function and Model Selectionmentioning

confidence: 99%

Product Design Time Forecasting by Kernel-Based Regression with Gaussian Distribution Weights

Shang

Yan

2016

Entropy

View full text Add to dashboard Cite

show abstract

“…Note that in deriving the exact bound in Appendix A, we assumed that the separating hyperplane u T x + v = 0 correctly separates the linearly separable training points; consequently, no other constraints are present in the optimization problem (6).…”

Section: The Linear Minimal Complexity Machinementioning

confidence: 99%

“…We begin with the optimization problem in (6), which was obtained from the exact bound on γ derived in Appendix A. In deriving the exact bound in Appendix A, we assumed that the separating hyperplane u T x + v = 0 correctly separates the linearly separable training points; hence, no other constraints are present in the optimization problem (6). For the convenience of the reader, (6) [also (48)] is reproduced below.…”

Section: Appendix B the Hard Margin MCM Formulationmentioning

confidence: 99%

“…The most commonly used variants are the maximum margin L 1 norm SVM [1], and the least squares SVM (LSSVM) [2], both of which require the solution of a quadratic programming problem. In the last few years, SVMs have been applied to a number of applications to obtain cutting edge performance; novel uses have also been devised, where their utility has been amply demonstrated [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24]. SVMs were motivated by the celebrated work of Vapnik and his colleagues on generalization, and the complexity of learning.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning a hyperplane classifier by minimizing an exact bound on the VC dimension1

Jayadeva

2015

Neurocomputing

View full text Add to dashboard Cite

The VC dimension measures the complexity of a learning machine, and a low VC dimension leads to good generalization. While SVMs produce state-of-the-art learning performance, it is well known that the VC dimension of a SVM can be unbounded; despite good results in practice, there is no guarantee of good generalization. In this paper, we show how to learn a hyperplane classifier by minimizing an exact, or Θ bound on its VC dimension. The proposed approach, termed as the Minimal Complexity Machine (MCM), involves solving a simple linear programming problem. Experimental results show, that on a number of benchmark datasets, the proposed approach learns classifiers with error rates much less than conventional SVMs, while often using fewer support vectors. On many benchmark datasets, the number of support vectors is less than one-tenth the number used by SVMs, indicating that the MCM does indeed learn simpler representations.

show abstract

“…The solution of LS-SVM lacks sparseness, which causes very slow test speed. There are some fast algorithms for LS-SVM, such as a CG algorithm [34,176] and an SMO algorithm [95], and a coordinate-descent algorithm [112]. These algorithms achieve low complexity, but their solutions are not sparse.…”

Section: Least-squares Svmsmentioning

confidence: 99%

Support Vector Machines

Du¹,

Swamy

2013

Neural Networks and Statistical Learning

View full text Add to dashboard Cite

SVM [12,201] is one of the most popular nonparametric classification algorithms. It is optimal and is based on computational learning theory [200,202]. The goal of SVM is to minimize the VC dimension by finding the optimal hyperplane between classes, with the maximal margin, where the margin is defined as the distance of the closest point in each class to the separating hyperplane. It has a general-purpose linear learning algorithm and a problem-specific kernel that computes the inner product of input data points in a feature space. The key idea of SVM is to project the training set in a high-dimensional space into a lower dimensional feature space by means of a set of nonlinear kernel functions, where the projections of the training examples are always linearly separable in the feature space. The hippocampus, a brain region critical for learning and memory processes, has been reported to possess pattern separation function similar to SVM [6].SVM is a three-layer feedforward network. It implements the structural riskminimization (SRM) principle that minimizes the upper bound of the generalization error. This induction principle is based on the fact that the generalization error is bounded by the sum of a training error and a confidence-interval term that depends on the VC dimension. Generalization errors of SVMs are not related to the input dimensionality, but to the margin with which it separates the data. Instead of minimizing the training error, SVM purports to minimize an upper bound of the generalization error and maximizes the margin between a separating hyperplane and the training data.SVM is a universal approximator for various kernels [70]. It is popular for classification, regression, and clustering. One of the main features of SVM is the absence of local minima. SVM is defined in terms of a subset of the learning data, called support vectors. It is a sparse representation of the training data, and allows the extraction of a condensed dataset based on the support vectors.Kernel

show abstract

A fast iterative single data approach to training unconstrained least squares support vector machines

Cited by 9 publications

References 33 publications

Product Design Time Forecasting by Kernel-Based Regression with Gaussian Distribution Weights

Product Design Time Forecasting by Kernel-Based Regression with Gaussian Distribution Weights

Learning a hyperplane classifier by minimizing an exact bound on the VC dimension1

Support Vector Machines

Contact Info

Product

Resources

About