Weighted least squares training of support vector classifiers leading to compact and adaptive schemes

Navia-Vázquez, Ángel; Pérez-Cruz, Fernando; Artés‐Rodríguez, Antonio; Figueiras-Vidal, Anı́bal R.

doi:10.1109/72.950134

Cited by 75 publications

(49 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An important difference with pruning methods in classical neural networks (Bishop, 1995;Hassibi & Stork, 1993;Le Cun, Denker, & Solla, 1990), e.g., optimal brain damage and optimal brain surgeon, is that in the LS-SVM pruning procedure no inverse of a Hessian matrix has to be computed. The LS-SVM pruning procedure can also be related to Interior Point and IRWLS methods for SVMs (Navia-Vázquez et al, 2001;Smola, 1999), where a linear system of the same form as (10) is solved in each iteration step until the conditions for optimality and the resulting sparseness property of the SVM are obtained. In each step of the IRWLS solution the whole training set is still taken into account and the sparse SVM solution is obtained after convergence.…”

Section: Sparse Approximation Using Ls-svmsmentioning

confidence: 99%

“…The QP-problem of the corresponding SVM formulation is typically solved by Interior Point (IP) methods (Cristianini & Shawe-Taylor, 2000;Smola, 1999), Sequential Minimal Optimization (SMO) (Platt, 1998) and iteratively reweighted least squares approaches (IRWLS) (Navia-Vázquez et al, 2001), while LS-SVMs (Suykens & Vandewalle, 1999b;Suykens et al, 2002;Van Gestel et al, 2001Viaene et al, 2001) result into a set of linear equations. Efficient iterative methods for solving large scale linear systems are available in numerical linear algebra (Golub & Van Loan, 1989).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Benchmarking Least Squares Support Vector Machine Classifiers

et al. 2004

View full text Add to dashboard Cite

Abstract. In Support Vector Machines (SVMs), the solution of the classification problem is characterized by a (convex) quadratic programming (QP) problem. In a modified version of SVMs, called Least Squares SVM classifiers (LS-SVMs), a least squares cost function is proposed so as to obtain a linear set of equations in the dual space. While the SVM classifier has a large margin interpretation, the LS-SVM formulation is related in this paper to a ridge regression approach for classification with binary targets and to Fisher's linear discriminant analysis in the feature space. Multiclass categorization problems are represented by a set of binary classifiers using different output coding schemes. While regularization is used to control the effective number of parameters of the LS-SVM classifier, the sparseness property of SVMs is lost due to the choice of the 2-norm. Sparseness can be imposed in a second stage by gradually pruning the support value spectrum and optimizing the hyperparameters during the sparse approximation procedure. In this paper, twenty public domain benchmark datasets are used to evaluate the test set performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances. These SVM and LS-SVM performances are consistently very good when compared to a variety of methods described in the literature including decision tree based algorithms, statistical algorithms and instance based learning methods. We show on ten UCI datasets that the LS-SVM sparse approximation procedure can be successfully applied.

show abstract

Section: Sparse Approximation Using Ls-svmsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Benchmarking Least Squares Support Vector Machine Classifiers

et al. 2004

View full text Add to dashboard Cite

show abstract

“…This is one of the main computational problems of these algorithms that prevent their application in very large speech databases. However, some solutions are already being developed [36][37][38].…”

Section: Is a Nonlinear Function Which Maps Vectormentioning

confidence: 99%

Robust ASR using Support Vector Machines

Solera-Ureña

Martín-Iglesias

Gallardo-Antolín

et al. 2007

Speech Communication

View full text Add to dashboard Cite

The improved theoretical properties of Support Vector Machines with respect to other machine learning alternatives due to their max-margin training paradigm have led us to suggest them as a good technique for robust speech recognition. However, important shortcomings have had to be circumvented, the most important being the normalisation of the time duration of different realisations of the acoustic speech units.In this paper, we have compared two approaches in noisy environments: first, a hybrid HMM-SVM solution where a fixed number of frames is selected by means of an HMM segmentation and second, a normalisation kernel called Dynamic Time Alignment Kernel (DTAK) first introduced in [1] and based on DTW (Dynamic Time Warping). Special attention has been paid to the adaptation of both alternatives to noisy environments, comparing two types of parameterisations and performing suitable feature normalisation operations. The results show that the DTA Kernel provides important advantages over the baseline HMM system in medium to bad noise conditions, also outperforming the results of the hybrid system.

show abstract

“…The idea can also be extended easily to trained fusion schemes based on other classifiers. Worth noting, sequential algorithms to solve the SVM optimization problem in (2), (3) have already been proposed (Navia-Vazquez et al, 2001), and can be used to extend the proposed idea, first constructing the user-independent solution and then refining it by incorporating the local data.…”

Section: Global Local and Adapted Fusion Schemesmentioning

confidence: 99%

Adapted user-dependent multimodal biometric authentication exploiting general information

Fiérrez-Aguilar

Garcia‐Romero

Ortega‐Garcia

et al. 2005

Pattern Recognition Letters

View full text Add to dashboard Cite

This is the author’s version of a work that was accepted for publication in Pattern Recognition Letters. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Pattern Recognition Letters 26.16 (2005): 2628 – 2639, DOI: 10.1016/j.patrec.2005.06.008A novel adapted strategy for combining general and user-dependent knowledge at the decision-level in multimodal biometric authentication is presented. User- independent, user-dependent, and adapted fusion and decision schemes are com- pared by using a bimodal system based on ¯ngerprint and written signature. The adapted approach is shown to outperform the other strategies considered in this pa- per. Exploiting available information for training the fusion function is also shown to be better than using existing information for post-fusion trained decisions.This work has been supported by the Spanish Ministry for Science and Tech- nology under projects TIC2003-09068-C02-01 and TIC2003-08382-C05-01

show abstract

Weighted least squares training of support vector classifiers leading to compact and adaptive schemes

Cited by 75 publications

References 20 publications

Benchmarking Least Squares Support Vector Machine Classifiers

Benchmarking Least Squares Support Vector Machine Classifiers

Robust ASR using Support Vector Machines

Adapted user-dependent multimodal biometric authentication exploiting general information

Contact Info

Product

Resources

About