Improved Training of An optimal Sparse Least Squares Support Vector Machine

Xia, Xiao Lei; Li, Kang; Irwin, G.W.

doi:10.3182/20080706-5-kr-1001.01191

Cited by 6 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Arbitrarily given for the th hidden node, the number of FPOs to evaluate the cost function and the number of FPOs to compute both the gradient vector and the Hessian matrix are given as follows: (53) Then, the number of FPOs for an epoch of search is (54) where denotes the average number of trial tests for the optimal update step.…”

Section: A Stage I-continuous Forward Rbf Neural Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Two-Stage Mixed Discrete–Continuous Identification of Radial Basis Function (RBF) Neural Models for Nonlinear Systems

Jia

Bai

2009

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

The identification of nonlinear dynamic systems using radial basis function (RBF) neural models is studied in this paper. Given a model selection criterion, the main objective is to effectively and efficiently build a parsimonious compact neural model that generalizes well over unseen data. This is achieved by simultaneous model structure selection and optimization of the parameters over the continuous parameter space. It is a mixed-integer hard problem, and a unified analytic framework is proposed to enable an effective and efficient two-stage mixed discrete-continuous identification procedure. This novel framework combines the advantages of an iterative discrete two-stage subset selection technique for model structure determination and the calculus-based continuous optimization of the model parameters. Computational complexity analysis and simulation studies confirm the efficacy of the proposed algorithm.

show abstract

Section: A Stage I-continuous Forward Rbf Neural Modelingmentioning

confidence: 99%

“…If the kernel function is chosen to be a Gaussian, then the hypotheses in SVMs are RBF networks [52]. For some SVM variants such as the least squares SVMs [49], the design of Gaussian RBF kernel is vital in improving the sparseness of the solution [22], [23], [50], [53].…”

Section: Introductionmentioning

confidence: 99%

Two-Stage Mixed Discrete–Continuous Identification of Radial Basis Function (RBF) Neural Models for Nonlinear Systems

Jia

Bai

2009

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

show abstract

“…Equation (3) shows that an LS-SVM can also be viewed as a ridge regression model. And the optimality conditions (7) indicate that the introduction of slack variable is the root of the nonsparseness problem, since in regression applications, most slack variables end as nonzero [20]. But the introduction of seems inevitable for the representation of training cost and thus the penalty parameter to indicate the tradeoff between training cost and generalization abilities.…”

Section: Least Squares Approximation Sparse Svmmentioning

confidence: 99%

A Novel Sparse Least Squares Support Vector Machines

Xia

Jiao

et al. 2013

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

The solution of a Least Squares Support Vector Machine (LS-SVM) suffers from the problem of nonsparseness. The Forward Least Squares Approximation (FLSA) is a greedy approximation algorithm with a least-squares loss function. This paper proposes a new Support Vector Machine for which the FLSA is the training algorithm—the Forward Least Squares Approximation SVM (FLSA-SVM). A major novelty of this new FLSA-SVM is that the number of support vectors is the regularization parameter for tuning the tradeoff between the generalization ability and the training cost. The FLSA-SVMs can also detect the linear dependencies in vectors of the input Gramian matrix. These attributes together contribute to its extreme sparseness. Experiments on benchmark datasets are presented which show that, compared to various SVM algorithms, the FLSA-SVM is extremely compact, while maintaining a competitive generalization ability.

show abstract

“…The implementation technique for the regression model has benefited from the recently-emerged training algorithm of the least-squares Fig. 2 Simplifying a three class DB2 tree to a single node support vector machine (LS-SVM) (Suykens and Vandewalle 1999;Gestel et al 2004) which is assured to have a sparse solution (Xia et al 2008). Meanwhile, the rest of the nodes in the decision tree each accommodates a standard binary SVM classifier trained on two contrasting classes.…”

Section: Introductionmentioning

confidence: 99%

A hierarchical multiclass support vector machine incorporated with holistic triple learning units

Xia

Irwin

2010

Soft Comput

Self Cite

View full text Add to dashboard Cite

This paper proposes a new hierarchical learning structure, namely the holistic triple learning (HTL), for extending the binary support vector machine (SVM) to multi-classification problems. For an N-class problem, a HTL constructs a decision tree up to a depth of dN=3e þ 1. A leaf node of the decision tree is allowed to be placed with a holistic triple learning unit whose generalisation abilities are assessed and approved. Meanwhile, the remaining nodes in the decision tree each accommodate a standard binary SVM classifier. The holistic triple classifier is a regression model trained on three classes, whose training algorithm is originated from a recently proposed implementation technique, namely the least-squares support vector machine (LS-SVM). A major novelty with the holistic triple classifier is the reduced number of support vectors in the solution. For the resultant HTL-SVM, an upper bound of the generalisation error can be obtained. The time complexity of training the HTL-SVM is analysed, and is shown to be comparable to that of training the oneversus-one (1-vs.-1) SVM, particularly on small-scale datasets. Empirical studies show that the proposed HTL-SVM achieves competitive classification accuracy with a reduced number of support vectors compared to the popular 1-vs-1 alternative.

show abstract

Improved Training of An optimal Sparse Least Squares Support Vector Machine

Cited by 6 publications

References 8 publications

Two-Stage Mixed Discrete–Continuous Identification of Radial Basis Function (RBF) Neural Models for Nonlinear Systems

Two-Stage Mixed Discrete–Continuous Identification of Radial Basis Function (RBF) Neural Models for Nonlinear Systems

A Novel Sparse Least Squares Support Vector Machines

A hierarchical multiclass support vector machine incorporated with holistic triple learning units

Contact Info

Product

Resources

About