A Fast SVM Training Algorithm

Dong, Jianxiong; Krzyżak, Adam; Suen, Ching Y.

doi:10.1007/3-540-45665-1_5

Cited by 18 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The feature vectors were scaled before they were used in classifiers in the training and testing stages [19], [22]. RAM., Each method was implemented using the C code.…”

Section: -Concavity Typesmentioning

confidence: 99%

See 1 more Smart Citation

Offline Handwritten Numeral Recognition Using Multiple Features and SVM classifier

Kim

Park²

2015

Journal of IKEEE

View full text Add to dashboard Cite

In this paper, we studied the use of the foreground and background features and SVM classifier to improve the accuracy of offline handwritten numeral recognition. The foreground features are two directional features: directional gradient feature by Kirsch operators and directional stroke feature by local shrinking and expanding operations, and the background feature is concavity feature which is extracted from the convex hull of the numeral, where the concavity feature functions as complement to the directional features. During classification of the numeral, these three features are combined to obtain good discrimination power. The efficiency of our scheme is tested by recognition experiments on the handwritten numeral database CENPARMI, where SVM classifier with RBF kernel is used. The experimental results show the usefulness of our scheme and recognition rate of 99.10% is achieved.

show abstract

“…The feature vectors were scaled before they were used in classifiers in the training and testing stages [19], [22]. RAM., Each method was implemented using the C code.…”

Section: -Concavity Typesmentioning

confidence: 99%

“…In the "one against the rest" strategy, k In SVM implementation, the optimization method should be used to solve the quadratic programming problem by using training samples. We used the SVM library [21], which is a fast SVM training algorithm that deals with large scale data [22].…”

mentioning

confidence: 99%

Offline Handwritten Numeral Recognition Using Multiple Features and SVM classifier

Kim

Park²

2015

Journal of IKEEE

View full text Add to dashboard Cite

show abstract

“…To measure the relative performance of different ISDAs, we ran all the algorithms with RBF Gaussian kernels on a MNIST dataset with 576-dimensional inputs (Dong et al, 2003), and compared the performance of our ISDAs with LIBSVM V2.4 (Chang et al, 2003) which is one of the fastest and the most popular SVM solvers at the moment based on the SMO type of an algorithm. The MNIST dataset consists of 60,000 training and 10,000 test data pairs.…”

Section: Performance Of the Iterative Single Data Algorithm And Compamentioning

confidence: 99%

Iterative Single Data Algorithm for Training Kernel Machines from Huge Data Sets: Theory and Performance

Kecman¹,

Vogt²

2005

Studies in Fuzziness and Soft Computing

View full text Add to dashboard Cite

Abstract.The chapter introduces the latest developments and results of Iterative Single Data Algorithm (ISDA) for solving large-scale support vector machines (SVMs) problems. First, the equality of a Kernel AdaTron (KA) method (originating from a gradient ascent learning approach) and the Sequential Minimal Optimization (SMO) learning algorithm (based on an analytic quadratic programming step for a model without bias term b) in designing SVMs with positive definite kernels is shown for both the nonlinear classification and the nonlinear regression tasks. The chapter also introduces the classic Gauss-Seidel (GS) procedure and its derivative known as the successive over-relaxation (SOR) algorithm as viable (and usually faster) training algorithms. The convergence theorem for these related iterative algorithms is proven. The second part of the chapter presents the effects and the methods of incorporating explicit bias term b into the ISDA. The algorithms shown here implement the single training data based iteration routine (a.k.a. per-pattern learning). This makes the proposed ISDAs remarkably quick. The final solution in a dual domain is not an approximate one, but it is the optimal set of dual variables which would have been obtained by using any of existing and proven QP problem solvers if they only could deal with huge data sets. IntroductionOne of the mainstream research fields in learning from empirical data by support vector machines (SVMs), and solving both the classification and the regression problems, is an implementation of the incremental learning schemes when the training data set is huge. The challenge of applying SVMs on huge data sets comes from the fact that the amount of computer memory required for a standard quadratic programming (QP) solver grows exponentially as the size of the problem increased. Among several candidates that avoid the use of standard QP solvers, the two learning approaches which recently have drawn the attention are the Iterative Single Data Algorithms (ISDAs), and the sequential minimal optimization (SMO) (Platt, 1998(Platt, , 1999Vogt 2002;Kecman, Vogt, Huang 2003;Huang and Kecman 2004).The ISDAs work on one data point at a time (per-pattern based learning) towards the optimal solution. The Kernel AdaTron (KA) is the earliest ISDA for SVMs, which uses kernel functions to map data into SVMs' high dimensional feature space (Frieß et al. 1998) and performs AdaTron learning (Anlauf and Biehl 1989) in the feature space. The Platt's SMO algorithm is an extreme case of the decomposition methods developed in (Osuna, Freund, Girosi 1997;Joachims 1999), which works on a working set of two data points at a time. Because of the fact that the solution for working set of two can be found analytically, SMO algorithm does not invoke standard QP solvers. Due to its analytical foundation the SMO approach is particularly popular and at the moment the widest used, analyzed and still heavily developing algorithm. At the same time, the KA although providing similar results in solving classific...

show abstract

“…The individual optimization steps can be run in parallel. -A fast SVM algorithm, which uses caching, digest and shrinking policies is given in [23]. -The clustering-based SVM [24] is a learning method that scans the data set before training the SVM.…”

Section: Parallel Support Vector Machine Approachesmentioning

confidence: 99%

HyParSVM – A New Hybrid Parallel Software for Support Vector Machine Learning on SMP Clusters

Eitrich¹,

Frings²,

Lang

2006

Euro-Par 2006 Parallel Processing

View full text Add to dashboard Cite

Abstract. In this paper we describe a new hybrid distributed/shared memory parallel software for support vector machine learning on large data sets. The support vector machine (SVM) method is a well-known and reliable machine learning technique for classification and regression tasks. Based on a recently developed shared memory decomposition algorithm for support vector machine classifier design we increased the level of parallelism by implementing a cross validation based on message passing. With this extention we obtained a flexible parallel SVM algorithm that can be used on high-end machines with SMP architectures to process the large data sets that arise more and more in bioinformatics and other fields of research.

show abstract

A Fast SVM Training Algorithm

Cited by 18 publications

References 12 publications

Offline Handwritten Numeral Recognition Using Multiple Features and SVM classifier

Offline Handwritten Numeral Recognition Using Multiple Features and SVM classifier

Iterative Single Data Algorithm for Training Kernel Machines from Huge Data Sets: Theory and Performance

HyParSVM – A New Hybrid Parallel Software for Support Vector Machine Learning on SMP Clusters

Contact Info

Product

Resources

About