Training linear SVMs in linear time

Joachims, Thorsten

doi:10.1145/1150402.1150429

Cited by 1,456 publications

(1,150 citation statements)

References 21 publications

Supporting

Mentioning

1,136

Contrasting

Unclassified

Order By: Relevance

“…The implementation of H-M 3 and HMSVM were provided by the authors while the implementation of the baseline algorithms (SVM and H-SVM ) and the versions in which binary classifiers are not independent (SVM l ∆ and H-SVM l ∆ ) were done modifying slightly Joachims' SVM perf [20]; this SVM implementation provided us with an excellent base due to its linear complexity. In all cases, when it was needed, we set the regularization parameter C = 1 and we used a linear kernel.…”

Section: Resultsmentioning

confidence: 99%

A semi-dependent decomposition approach to learn hierarchical classifiers

Díez

Coz

Bahamonde

2010

Pattern Recognition

View full text Add to dashboard Cite

In hierarchical classification, classes are arranged in a hierarchy represented by a tree or a forest, and each example is labeled with a set of classes located on paths from roots to leaves or internal nodes. In other words, both multiple and partial paths are allowed. A straightforward approach to learn a hierarchical classifier, usually used as a baseline method, consists in learning one binary classifier for each node of the hierarchy; the hierarchical classifier is then obtained using a top-down evaluation procedure. The main drawback of this naïve approach is that these binary classifiers are constructed independently, when it is clear that there are dependencies between them that are motivated by the hierarchy and the evaluation procedure employed. In this paper, we present a new decomposition method in which each node classifier is built taking into account other classifiers, its descendants, and the loss function used to measure the goodness of hierarchical classifiers. Following a bottom-up learning strategy, the idea is to optimize the loss function at every subtree assuming that all classifiers are known except the one at the root. Experimental results show that the proposed approach has accuracies comparable to state-of-the-art hierarchical algorithms and is better than the naïve baseline method described above. Moreover, the benefits of our proposal include the possibility of parallel implementations, as well as the use of all available well-known techniques to tune binary classification SVMs.

show abstract

Section: Resultsmentioning

confidence: 99%

A semi-dependent decomposition approach to learn hierarchical classifiers

Díez

Coz

Bahamonde

2010

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…Compute (k) by formula (11). if (k) = 0 then Sort the indices set {1, · · · , l} by the values of (k) in increasing order.…”

Section: Algorithm 2 the First-order Working Set Selection Rulementioning

confidence: 99%

“…Several researchers also explore how to train the primal form of (4) and the extended models fast. The existing algorithms can be broadly categorized into two categories: the cutting-plane methods [11,5,12,13,25], and subgradient methods [3,17]. For example, in [17], Shalev-Shwartz et al described and analyzed a simple and effective stochastic sub-gradient descent algorithm and prove that the number of iterations required to obtain a solution of accuracy is O(1/ ).…”

Section: Introductionmentioning

confidence: 99%

Two New Decomposition Algorithms for Training Bound-Constrained Support Vector Machines*

Niu

Zhou

Zhao

et al. 2015

Foundations of Computing and Decision Sciences

View full text Add to dashboard Cite

Abstract. Bound-constrained Support Vector Machine(SVM) is one of the stateof-art model for binary classification. The decomposition method is currently one of the major methods for training SVMs, especially when the nonlinear kernel is used. In this paper, we proposed two new decomposition algorithms for training bound-constrained SVMs. Projected gradient algorithm and interior point method are combined together to solve the quadratic subproblem efficiently. The main difference between the two algorithms is the way of choosing working set. The first one only uses first order derivative information of the model for simplicity. The second one incorporate part of second order information into the process of working set selection, besides the gradient. Both algorithms are proved to be global convergent in theory. New algorithms is compared with the famous package BSVM. Numerical experiments on several public data sets validate the efficiency of the proposed methods.

show abstract

“…SMO, proposed by Platt [7], is another algorithm that trains SVM efficiently which decomposes the overall QP problem into sub-problems and chooses to solve the smallest possible optimization problem at every step. Joachims [8] presented a Cutting Plane algorithm which is used to train Linear SVM in a linear time for classification task. In [9] a fast algorithm for solving linear SVMs with loss function toward data mining tasks for large data sets was developed.…”

Section: Related Workmentioning

confidence: 99%

Accelerating the Training Process of Support Vector Machines by Random Partition

Xu¹,

Li²,

Li³

et al. 2014

IJCTE

View full text Add to dashboard Cite

Abstract-In this paper we present a novel method, Random Partition based SVM (RPSVM), for speeding up SVM training. Instead of clustering the training data prior to training, RPSVM randomly partitions the training data into several clusters and then uses the centers of the clusters to train an initial SVM. This trained SVM is used to find critical clusters which are located on the decision boundary. The same procedure is applied repeatedly to each of the critical clusters, resulting in a refined SVM which consists of the supporting vectors in the initial round of training and those in the repeated round. This procedure is repeated recursively until no critical cluster exists, resulting in the final SVM. Our experiments on synthetic and real data sets have shown that RPSVM is indeed scalable to large data sets while the high performance is retained.

show abstract

Training linear SVMs in linear time

Cited by 1,456 publications

References 21 publications

A semi-dependent decomposition approach to learn hierarchical classifiers

A semi-dependent decomposition approach to learn hierarchical classifiers

Two New Decomposition Algorithms for Training Bound-Constrained Support Vector Machines*

Accelerating the Training Process of Support Vector Machines by Random Partition

Contact Info

Product

Resources

About