SMO-Based Pruning Methods for Sparse Least Squares Support Vector Machines

Zeng, Xiangyan; Chen, Xuewen

doi:10.1109/tnn.2005.852239

Cited by 85 publications

(41 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pruning in LS-SVM is investigated using an SMO-based pruning method [226]. It requires solving a set of linear equations for pruning each sample, causing a large computational cost.…”

Section: Pruning Svmsmentioning

confidence: 99%

Support Vector Machines

Du¹,

Swamy

2013

Neural Networks and Statistical Learning

View full text Add to dashboard Cite

SVM [12,201] is one of the most popular nonparametric classification algorithms. It is optimal and is based on computational learning theory [200,202]. The goal of SVM is to minimize the VC dimension by finding the optimal hyperplane between classes, with the maximal margin, where the margin is defined as the distance of the closest point in each class to the separating hyperplane. It has a general-purpose linear learning algorithm and a problem-specific kernel that computes the inner product of input data points in a feature space. The key idea of SVM is to project the training set in a high-dimensional space into a lower dimensional feature space by means of a set of nonlinear kernel functions, where the projections of the training examples are always linearly separable in the feature space. The hippocampus, a brain region critical for learning and memory processes, has been reported to possess pattern separation function similar to SVM [6].SVM is a three-layer feedforward network. It implements the structural riskminimization (SRM) principle that minimizes the upper bound of the generalization error. This induction principle is based on the fact that the generalization error is bounded by the sum of a training error and a confidence-interval term that depends on the VC dimension. Generalization errors of SVMs are not related to the input dimensionality, but to the margin with which it separates the data. Instead of minimizing the training error, SVM purports to minimize an upper bound of the generalization error and maximizes the margin between a separating hyperplane and the training data.SVM is a universal approximator for various kernels [70]. It is popular for classification, regression, and clustering. One of the main features of SVM is the absence of local minima. SVM is defined in terms of a subset of the learning data, called support vectors. It is a sparse representation of the training data, and allows the extraction of a condensed dataset based on the support vectors.Kernel

show abstract

“…Pruning in LS-SVM is investigated using an SMO-based pruning method [226]. It requires solving a set of linear equations for pruning each sample, causing a large computational cost.…”

Section: Pruning Svmsmentioning

confidence: 99%

Support Vector Machines

Du¹,

Swamy

2013

Neural Networks and Statistical Learning

View full text Add to dashboard Cite

show abstract

“…The sparseness is imposed by subsequently omitting data that introduce the smallest training errors and retraining the remaining data. In the following, the SMO-Based pruning Algorithms are given in [7]. From (5), the Karush-Kuhn-Tucker (KKT) conditions for optimality are:…”

Section: Smo-based Pruning Algorithms For Ls-svmmentioning

confidence: 99%

“…Iterative retraining requires more intensive computations than training a single non-sparse LS-SVM. This paper we use the methods proposed in [7]. Both the computational costs and regression accuracy are solved.…”

Section: Introductionmentioning

confidence: 99%

Nonlinear Systems Modeling Using LS-SVM with SMO-Based Pruning Methods

Sun

Song

et al. 2007

Advances in Neural Networks – ISNN 2007

View full text Add to dashboard Cite

Abstract. This paper firstly provides a short introduction to least square support vector machine (LS-SVM), then provides sequential minimal optimization (SMO) based on Pruning Algorithms for LS-SVM, and uses LS-SVM to model nonlinear systems. Simulation experiments are performed and indicated that the proposed method provides satisfactory performance with excellent accuracy and generalization property and achieves superior performance to the conventional method based on common LS-SVM and neural networks.

show abstract

“…For more on LSSVM pruning algorithms, Hoegaerts et al [12] provided a comparison among these algorithms and concluded that pruning schemes can be divided into QR decomposition and searching feature vector. Instead of determining the pruning points by errors, Zeng and Chen [13] introduced the sequential minimal optimization method to omit the datum that will lead to minimum changes to a dual objective function. Based on kernel partial least squares identification, Song and Gui [14] presented a method to get base vectors via reducing the kernel matrix by Schmidt orthogonalization.…”

Section: Introductionmentioning

confidence: 99%

Reconstruct the Support Vectors to Improve LSSVM Sparseness for Mill Load Prediction

Shi

Guo

et al. 2017

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The sparse strategy plays a significant role in the application of the least square support vector machine (LSSVM), to alleviate the condition that the solution of LSSVM is lacking sparseness and robustness. In this paper, a sparse method using reconstructed support vectors is proposed, which has also been successfully applied to mill load prediction. Different from other sparse algorithms, it no longer selects the support vectors from training data set according to the ranked contributions for optimization of LSSVM. Instead, the reconstructed data is obtained first based on the initial model with all training data. Then, select support vectors from reconstructed data set according to the location information of density clustering in training data set, and the process of selecting is terminated after traversing the total training data set. Finally, the training model could be built based on the optimal reconstructed support vectors and the hyperparameter tuned subsequently. What is more, the paper puts forward a supplemental algorithm to subtract the redundancy support vectors of previous model. Lots of experiments on synthetic data sets, benchmark data sets, and mill load data sets are carried out, and the results illustrate the effectiveness of the proposed sparse method for LSSVM.

show abstract

SMO-Based Pruning Methods for Sparse Least Squares Support Vector Machines

Cited by 85 publications

References 14 publications

Support Vector Machines

Support Vector Machines

Nonlinear Systems Modeling Using LS-SVM with SMO-Based Pruning Methods

Reconstruct the Support Vectors to Improve LSSVM Sparseness for Mill Load Prediction

Contact Info

Product

Resources

About